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PREFACE 



All the major activities conducted by administrators, faculty, and student personnel workers in 
postsecondary education institutions are, presumably, aimed toward bringing about certain 
kinds of "educational outcomes." The activities of planning, budgeting, management, 
evaluation, research, scholarship, instruction, and other developmental activities for students, 
and even staff development, all have educational outcomes as a primary focus. Other groups 
within an educational institution— such as, the secretarial staff and the computer operators- 
are also expected to contribute to various educational outcomes. 

In this "age of accountability" administrators and others have been especially concerned 
about educational outcomes and their measurement. Postsecondary institutions are being 
faced with increasing pressures to define better "what they are al I about," and to communicate 
better their purposes and goals to many different audiences. Institutions are also being called 
on to provide factual evidence that they and their programs are providing the benefits that were 
intended, and that these outcomes are being produced in a cost-effective manner. Because of 
the important needs in the area of educational outcomes, NCHEMS has devoted major 
resources and efforts over the last five years toward developing procedures and materials to 
help postsecondary education institutions collect and use outcomes information. In 1973 an 
Inventory of Outcome Variables and Measures was published. In 1975, the Outcome Measures 
and Procedures Manual was published, giving standard definitions and alternative data 
acquisition procedures for approximately 50 measures that different postsecondary education 
decision makers (including state legislators) said were most important for their needs. The 
most recent development is the NCHEMS Outcomes Structure. 

This document presents an overview of the NCHEMS Outcomes Structure, and then provides 
some basic procedures for using the Structure that can have important, practical impacts for 
an institution, program, or other use context. The Outcomes Structure is a system designed to 
serve as a framework for organizing information in an effective way for purposes of 
classification, analysis, and decision making. It rests upon significant bodies of previous work, 
its practical application has been tested in a preliminary way at one public university and eight 
private colleges, it has been formally reacted to by a diverse group of over 100 people, and it is 
intended to continue its development as time passes. 

The current document is designed specifically for those who are trying to introduce and orient 
the people on their campus to the Outcomes Structure and its potential, and to train people in 
how to apply or use it. It also is expected to be of day-to-day use to administrators, faculty, and 
other decision makers In postsecondary education institutions. In addition, many of the 
general procedures presented here are believed to have some applicability to needs of program 
and resource planners at other levels within postsecondary education, for example, planners at 
the state and federal levels and planners for groups or systems of institutions. 

This is the second of two documents presenting the NCHEMS Outcomes Structure. The first 
one gives an in-depth discussion of the Structure, its rationale, its development, and its 
underlying empirical support/ It was prepared primarily for use by those wanting a detailed 
description and conceptual discussion of the Outcomes Structure and an in-depth 
presentation of the concept of an "educational outcome," including researchers and 
administrators or faculty specializing in outcomes. A number of those reviewing drafts of both 

' Oscar T Lenning, Yong S Lee, Sidney S Micek, and Allan L Service, A Structure for the Outcomes of Postsecondary Education (Boulder, Colo National 
Center for Higher Education Management Systems, 1977) 
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PREFACE (Continued) 



documents expressed the opinion that many to whom this current document is aimed, would 
also find the first document of interest and use as a supplemental source, and that it should be 
read prior to reading this document. 

The procedures discussed in this document are based on logic and limited, preliminary tests. 
Although these preliminary tests obtained a number of very positive results, the real value of 
the Structure will not be known until it receives widespread application. As experience with 
using the Structure in many different contexts occurs, a revised version of this document will 
be published to report any insights gained and any new procedures or techniques found to be 
useful. 

Many of the people acknowledged in the in-depth conceptual document referred to above also 
reviewed an earlier draft of this document, provided helpful suggestions, and in other ways 
contributed to the development of this document. Appreciation is hereby again extended to 
them, with special acknowledgment and thanks going to George Barton, Cathleen Bower, 
Edward Cooper, Jean Endo, Sidney Micek, Edward Myers, Robert Passmore, Nancy 
Renkiewicz, David Witmer, and participants in the CASC/NCHEMS project— including Task 
Force members who made campus visits and the campuses where the Structure was tried out. 



x 

10 



INTRODUCTION 



Having a wide variety of outcomes information without any structure is analogous to 
possessing a file cabinet in which the contents are arranged randomly. Similarly, without 
agreement on a common language and context for outcomes, information about outcomes can 
be misunderstood or be seriously misleading, and it becomes difficult for institutional officials 
to communicate succinctly how their institution and program differs from its counterparts. An 
effective outcomes structure provides such a common language. 

The sheer volume of potential outcomes information in postsecondary education poses a 
significant barrier to using outcome information in institutional planning and management. 
Although a number of attempts have been made to develop structural systems for organizing 
outcomes information, they have been inadequate for meeting the practical need for 
comprehensiveness and institution-wide use in postsecondary education. Because of the 
limitations in the state of the art and the important planning and management needs that exist 
in this area for postsecondary education, an NCHEMS project was conceived and executed 
that aimed to: (1) develop a system that could effectively organize information about the full 
range of potential postsecondary education outcomes for purposes of classification, analysis, 
and decision making, and (2) develop an extensive set of standard definitions for the different 
types of educational outcomes. After more than two years of concentrated work, an Outcomes 
Structure resulted that reviewers outside NCHEMS feel has potential for being of use to 
postsecondary education decision makers in such areas as identifying educational needs, 
developing goals, translating goals into more concrete objectives, evaluating institutions and 
their programs, and so forth. Furthermore, preliminary tests of the Structure at one public 
university and eight small private colleges were quite encouraging, as reported in the base 
document that gives in-depth information about the Structure, its development, and its 
underlying empirical support (Lenning, Lee, Micek, and Service, 1977). Nevertheless, it is 
expected that improvements will be made in the Structure as a result of its use within different 
types of institutions and various use contexts within those institutions. 

The purpose of this document is specifically to provide guidance for those wishing to orient 
people to the Structure and train them in its use. The first three parts provide information about 
the Structure and its use which should also be helpful to administrators, faculty, and others on 
campus who wish to try using the Outcomes Structure in a practical manner. Part I presents an 
overview of the Structure and its components. Part II describes in some detail the specific 
practical uses perceived for the Structure in postsecondary education planning, management, 
evaluation, and other contexts. Then, Part III lists step by-step procedures for accomplishing 
three basic processes that allow the Structure to be applied as indicated in Part II. Finally, Part 
IV— which is not aimed at users directly, as are the first three parts— assists trainers and others 
who wish to orient people on their campus to the Structure and its use. 
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PARTI 

AN OVERVIEW OF THE NCHEMS 
OUTCOMES STRUCTURE 



Postsecondary education outcomes are the end 
results of the processes that ocoui within 
postsecondary education institutions and 
programs. They include both the direct results of 
these processes and any short-term and long- 
term consequences of those direct results. 
Furthermore, some of them may be intended 
outcomes whife others are unintended— and 
while outcomes are generically neutral, people 
will be attaching positive and/or negative values 
to them. In using the NCHEMS Outcomes 
Structure, information about outcomes is 
organized according to three dimensions: (1) the 
persons, groups, or things that receive and/or 
are affected by the outcome(s) or concern, or that 
are intended to receive or be affected by it; 
(2) whether or not the outcome involves a change 
in status— maintenance versus change— and the 
basic, specific entity that is maintained or 



changed; and (3) when the outcome occurs or is 
intended or expected to occur. These dimensions 
have been named "audiei 3," "type-of-outcome," 
and "time," respective!), and each (except for 
time) has been assigned a number of specifically 
defined categories and subcategories, it is 
intended that those dimensions and categories 
not of serious concern to a particular user of the 
Structure will be ignored, or will be modified ano 
adapted to better meet local needs and 
situations. (The same is true of the various 
procedures for using the Structure that are 
described later in this document.) 

The Outcomes Structure is illustrated graphically 
below. For keeping straight the three dimensions, 
it may help to talk in terms of three Ws— who, 
what, and when. Who is receiving or being 
affected by what outcomes and when? 



AUDIENCE 
DIMENSION 

(Who or what receives 
or is affected by the 
outcome, or is intend- 
ed to receive or be 
affected by it?) 




TIME DIMENSION 

(When is the out- 
come expected to 
occur, or when does 
it occur?) 



TYPE OF OUTCOME DIMENSION 

(What basic entity is, or is intended to 
be, maintained or changed?) 
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The Audience Dimension 

Table 1 lists the major (first level of detail) 
categories of the "audience 1 ' dimension. Table 2 
shows subcategories (second-level categories) 
separately for each of the major categories. For 
those who need a shorthand for referring to 
specific categories or for developing filing 
systems for particular categories of outcomes in- 
formation, code numbers are also provided in 
Table 2 fty each category and subcategory. (The 
coding scheme for the Structure is discussed in 
detail in Appendix A.) For many applications of 
the Structure, particular subcategories of this 
dimension will need to be further subdivided into 
even finer categories— such as, dividing students 
in a program into those majoring in the program 
and those just taking courses in the program, or 
into disabled stude r k .s and nondisabled students. 
Such "extending the Structure," as this additional 
subdividing of categories of any of the dimen 
sions is called, depends on t r ne philosophy and 
purposes of the user of the Structure and on the 
context in which the application is taking place 



(for example, someone using it in curricular 
development will probably need different ad- 
ditional categories from someone using it in 
program administrative planning). Specific 
procedures for extending the Structure are pro- 
vided in Part III of this document. 



The Ty?3"Of"Outcome Dimension 

Table 3 piesents names, definitions, and code 
numbers for the five overall (first level of detail) 
categories of the "type-of-ouicome" dimension. 
Table 4 provides additional levels of detail by in- 
dicating names and code numbers for the 
categories and subcategories into which each of 
the five broadest categories has been divided. 
Standard definitions along with example outcome 
measures and indicators are provided in Appen- 
dix B for each category and subcategory iden- 
tified in Table 4. As with the audiei.ee dimension, 
the structure will often need to be extended on the 
type-of-outcome dimension. For example, in 
curriculum development it may be deshable to 



Table 1 

MAJOR CATEGORIES OF THE "AUDIENCE" DIMENSION 



10. Individual/Group Clients— This category refers to persons or groups of persons who are 
direct clients of the postsecondary education unit of concern and/or their immediate 
associates, such as family and relatives or peers. 

20. Interest Based Communities— /his category refers to large groups of people that are 
identified as entities working toward a well-defined interest or mission 

30. Geographic-Based Communities— This category refers to large groups of people 
defined on the basis of functional territorial boundaries. 



40. Aggregates of People— This category refers to subpopulations of people distinguished 
by particular characteristics that ma^ indicate common concerns, needs, or wants, but 
who do not necessarily have a con.mcn interest or mission, and therefore do not 
constitute communities. 

50. Ot her Audiences— Examples would be the natural environment that is affected by 
university-sponsored research (which in turn would be expected to have impacts on 
audiences such as individuals and communities) and populations of animals (such as 
the animals affected by efforts to keep depleted species from becoming extinct or by 
the development of veterinary medicines). 
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Table 2 



SUBCATEGORIES OF THE "AUDIENCE" DIMENSION 



10 Individual/Group Client -This category refers to persons or groups of persons who are direct ci writs of the postsecondary education unit of 
concern and for their immediate associates, such as family and relet Ives or peers 

1 1 Sfude/ifs— Individuals or groups of Individuals who currently are enrolled in the program, Institution, or system of postsecondary education 

12 For*<or Studen ft— Individuals or groups of Individuals who formerly were enrolled in the program, institution, or system of postsecondary 
education 

13 Fam//y and Relatives of Students or Former Students 

14 Rears and 4ssoc/sres of StuJents or Former Students 

15 Faculty 

1 6 Stiff Other then Faculty 

17 Other Individual /Group Clients— An example would be sn Individual who Is none of the above but Is served by an advisory service offered 



20 Interest-Based Communities— This category refers to large groups that are Identified as entitles working toward a well-defined interest or 
mission 

21 Private Enterprise Communities— Communities where a major purpose is f I, uncial remuneration and profit— for example, corporations, 
amaii businesses, and farmers 

22 Association Communities— Communities where members belong on tha basis of affiliation rather than employment, such as unions and 
professional societies 

23 Government Communities— Communities desigrsd to administer government regulations and services, such as cit-> hall, state department 
of education, and legislative communities 

24 Nongovernmental/ Public Service Communitias Other than the Institution Producing the Outcome— Nonprofit service organizations, such 
as schools, hospitals, welfare agencies, philanthropic foundations, colleges (other then the college producing the outcome), and research 
organizations. 

25 Institution or Institutional Unit Producing tha Outcome— The postsecondary education institution and /or units within that institution that 
are perceived as the producer/ facilitator of theoutcome(a) of concern 

20 Other Interest-Based Communities— An example would be an ad hoc coalition task force of representatives from two or more of the above 
areas. 

30 Geographic-Based Communities— This ca, sy refers to large groups defined on the basis of functional territorial boundarias 

31 Local Community— A township, city, county, metropolitan area, or other type of locality having particular boundaries It Is not necessarily 
restricted to the lecsi or jurisdictional boundary, but the functional one in which the Impact of the institution is (or should be) directly and 
physically felt Thi boundaries will vary with the institution/program and outcome of concern 

32 TheStata 

33 A Region— An aggregation of states or parts of states 

34 The Nation 

& An International C .nmunity 

30 Other Geographic-Based Communities— An example would be e research discovery that effects primerlly people liv.ng in the coldest 
latitudes, or where it snows heav'ty 

40 Aggregatas of People— lh\a category refers to eubpopuletions of people distinguished by particular charactaristics that may indicate common 
concerns, needs, or wants, but who do not necessarily have a common Imcf -»t or mission, end therefore do not constitute communities 

41 Ability Level Subpopulations— Subpopuletions defined according to level of ability /proficiency on general intellectual functioning or 
specific skills— for example, gifted, typical, disadvantaged, or skilled, semi-skilled, unskilled 

42 Age Subpopulations 

43 Educational Level Subpopulations 

44 Income Level Subpopulatior _ 

45 Occupation Subpopulations 

46 Physical Disability Conr' tion Subpopulations 

47 fUce Subpopulations 

48 Sex SubDopulations 
4° Other Such Aggregatas 

50 Other . ^diences— Examples would be the naturel environment that is affected by university-sponsored research (which in turn would be 
expected to have impacts on audiences such as individuals end communities) end populetions of animals (such as the enimais affected by 
efforts to keep depleted species from becoming extinct or by the development of veterinary medicines) 



by the college 
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divide "intellectual skills 11 (code number 2250) in- 
to the following categories of Bloom's Taxonomy 
(1956): ability to translate (code 22501), ability to 
interpret (code 22502), ability to extrapolate (code 
22503), ability to apply (code 22504), ability to 
analyze (code 22505), ability to synthesize (code 
22506), and ability to evaluate (code 22507V 

One thing should bp made clear. No program or 
institution can hope to focus on outcomes within 



all the categories of Table 4, or perhaps even on 
many of the categories. Rather, this is a universe 
of possible outcome categories from which one 
must pick and choose those that are most impor- 
tant, in the light of factors such as philosophy, 
resources available, and clientele needs. To try to 
devote primary effort toward accomplishing too 
many types of outcomes could have disastrous 
consequences. 



Table 3 

THE MAJOR CATEGORIES OF THE "TYPE-OF-OUTCOME" DIMENSION 



Category 
Code Number 


The Major "Type-of-Outcome" 
Category Names and Definitions 


1000 


Economic Outcomes— Maintenance or change in economic characteristics and 
conditions of individuals, groups, organizations, and communities, e.g., in 
economic access, in economic mobility and independence, in economic security, 
and in income and standard of living. 


2000 


numan onar acre/78wu uuicomes — Maintenance or onangu in nuriiaii makeup aim 
characteristics (other than knowledge and understanding) of individuals, groups, 
organizations, and communities, e.g., aspirations, competence and skills, affective 
characteristics, perceptual characteristics, physical and physiological character- 
istics, personality and personal coping characteristics, recognition and certifica- 
tion, and social roles. 


3000 


Knowledge, Technology, and Art Form Outcomes— Maintenance or change in the 
knowledge and understanding, technology, or the art forms and works possessed 
or mastered by individuals, groups, organizations, and communities, e.g., 
discoveries and inventions, technical developments, syntheses and reformulations 
of knowledge, new schools of thought in art and works created in those new 
traditions, renovation of art works. 


4000 


Resource and Service Provision Outcomes— Maintenance or change in the direct 
resources and services (other than those included above) provided to individuals, 
groups, organizations, and communities, e.g., providing facilities, events, 
advisory assistance, analytic assistance, teaching, health care, and leadership. 


5000 


Other Maintenance and Change Outcomes— Examples would be: maintenance or 
change in the format, arrangement, activity, or administrative operation of an 
organization or institution; maintenance or change in the aesthetic/cultural level 
of the local community; maintenance or change in family or community activities, 
practices, and traditions. 
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Table 4 

CODED LISTI NG OF THE SECON D- AN D TH I RD-LEVEL SU BCATEGORI ES 
FOR EACH FIRST-LEVEL CATEGORY OF THE TYPE-OF-OUTCOME DIMENSION 3 



C^rftmber Entltv Be,n 0 Maintained or Changed 


Category 

Code N umber Entity Being Maintained or Changed 


1000 ECONOMIC OUTCOMES 

1 1 00 Economic Access and Independence Outcomes 
1110 Economic Access 

1 1 20 Economic Flexibility, Adaptability, and Security 
1 1 30 Income and Standard of Living 

1200 Economic Resources and Costs 

1 21 0 Economic Costs and Efficiency 

1 220 Economic Resources (including employees) 

1300 Economic Production 

1310 Economic Productivity and Production 
1320 Economic Services Provided 

1400 Other Economic Outcomes 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 

2700 Power and/or Authority 

2770 Job, School, or Life Success 

2780 Other Status, Recognition, and Certification Outcomes 

2800 Social Activities and Roles 

281 0 Adjustment to Retirement 
2820 Affiliations 

2830 Avocationai and Social Activities and Roles 
2840 Career and Vocational Activities and Roles 
2850 Citizenship Activities and Roles 
2860 Famll/ Activities and Roies 
2870 Friendships and Relationships 
2880 Other Activity and Role Outcomes 

2900 Other Human Characteristic Outcomes 


2000 HUMAN CHARACTERISTIC OUTCOMES 

2100 Aspirations 

2110 Desires, Aims, and Goals 
21 20 Dislikes, Likes, and Interests 
2130 Motivation or Drive Level 
21 40 Other A spi rational Outcomes 

2200 Competence and Skills 
2210 Academic Skills 

2220 Citizenship and Family Membership Skills 
2230 Creativity Skills 
2240 Expression and Communication Skills 
2250 Intellectual Skiils 

2260 interpersonal, Leadership, and Organizational Skills 
2270 Occupational and Employability Skills 
2280 Physical and Motor Skills 
2290 Other Skill Outcomes 

2300 Morale, Satisfaction, and Affective Characteristics 
2310 Attitudes and Values 

2320 Beliefs, Commitments, and Philosophy of Life 
2330 Feelings and Emotions 
2340 Mores, Customs, and Standards of Conduct 
2350 Other Affinitive Outcomes 

2400 Perceptual Ch*, act eristics 

241 0 Perceptual Awareness and Sensitivity 
2420 Perception of Self 
2430 Perception of Others 
2440 Perception of Things 
2450 Other Perceptual Outcomes 

2500 Personality and Personal Coping Characteristics 
2510 Advent urousness and Initiative 
2520 Autonomy and Independence 
2530 Dependability and Responsibility 
2540 Dogmatic/Open-Minded, Authoritarian/Democratic 
2550 Flexibility and AdSDtabi I tv 
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2560 Habits 

2570 Psychological Functioning 
2580 Tolerance and Persistence 
2590 Other Personality and Personal Coping Outcomes 

2600 Physical and Physiological Characterises 
2610 Physical Fitness and Traits 
2620 Physiological Health 
2630 Other Pfc 'cal or Physiological Outcomes 

2700 Status, Recognition, and Certification 

2710 Completion or Achievement Awsrd 
2720 Credit Recognition 
2730 Image, Reputation, or Status 
2740 Licensing and Certification 

2750 Obtaining a Job or Admission to a Follow-up Program 


3000 KNOWLEDGE, TECHNOLOGY, AND ART FORM OUTCOMES 

3)C0 General Knowledge and Understanding 

31 10 Knowledge and Understanding of General Facts *nd 
Terminology 

3120 Knowledge and Understanding of General Processes 
31 30 Knowledge and Understanding o # General Theory 
3140 Other General Knowledge and Understanding 
3200 Specialized Knowledge and Understanding 

3210 Knowledge and Understanding of Specialized Facts 

and Terminology 
3220 Knowledge and Understanding of Specialized 

Processes 

3230 Knowledge and Understanding of Specialized Theory 
3240 Other Specialized Knowledge and Understanding 

3300 Research and Scholarship 

3310 Research and Scholarship Knowledge and 

Understanding 
3320 Research and Scholarship Products 

3400 Art Forms and Works 
3410 Architecture 
3420 Dance 

3430 bate and Oratory 
3440 Drama 

34b0 Literature and Writing 
3460 Music 

3470 Painting, Drawing, and Photography 
3480 Sculpture 
3490 Other Fine Arts 

3500 Other Knowledge, Technology, and Art Form Outcomes 


4000 RESOURCE A ND SERVICE PROVISION OUTCOMES 

4100 Provision of Facilities and Events 
41 (0 Provision of Facilities 
4120 Provision or Sponsorship of Events 

4200 Provision of Direct Services 
4210 Teaching 

4220 Advisory and Analytic Assistance 
*230 Treatment, Care, and Referral Services 
4240 Provision of Other Services 

4300 Other Resource and Service Provision Outcomes 


5000 OTHER MAINTENANCE AND CHANGE OUTCOMES 
5100 Aesthetic-Cultural Activities, Traditions, and Conditions 
5200 Organizational Format. Activity, and Operation 
5300 Other Maintenance and Change 



4 The fourth lava! categories, Into which any of the categories listed here can be divided, are "maintenance" (a fourth digit of "1")and "change" (a fourth digit of "2") 
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The Time Dimension 

No formal categories and subcategories were 
assigned to the time dimension because no time 
categories would apply across all audience 
categories. Furthermore, even at the broadest 
level, the time categories of interest would be ex- 
pected to vary depending on the philosophy of the 
user of the Structure and the particular need in his 
or her context. Time is so important in planning, 
however, that it was still made a dimension of the 
Structure— and the categories to use were left 
completely up to the user. 



To illustrate how specialized the taxonomy for the 
time dimension may need tc be, Ms. Jean Endo of 
the University of Colorado at Boulder reported to 
this writer on recommended times during the year 
for collecting student outcomes and related data 
using survey questionnaires— as based on their 
experiences— and when during the students' 
college and alumni career different data should 
be collected. These can be transformed into a 
'Taxonomy of Times to Consider for Collecting 
Undergraduate Student Outcomes Data Using 
Survey Questionnaires," as shown in Table 5. 



Tables 

TAXONOMY OFTIMES TO CONSIDER FOR COLLECTING UNDERGRADUATE 
STUDENT OUTCOMES DATA USING SURVEY QUESTIONNAIRES 3 



100 Data from Lower Division Students 
110 Data from Freshman Students 

111 Data Collected Prior to Fall Registration 

112 Data Collected One Month Following the First Day of Classes in the Fall 

113 Data Collected One Month Prior to First Semester Final Exams 

114 Data Collected One Month After Spring Semestrr Classes Begin 

115 Data Collected One Month Prior to Spring Semester Final Exams 

116 Other, for Example, in the Middle of the Summer Term 

120 Data from Sophomore Students 

121 Data Collected Priorto Fall Registration 

122 Data Collected One Month Following the First Day of Classes in the Fall 

123 Data Collected One Month Prior to First Semester Final Exams 

124 Data Collected One Month After Spring Semester Classes Begin 

125 Data Collected One Month Prior to Spring Semester Final Exams 

126 Other, for Example, in the Middle of the Summer Term 

200 Data from Upper Division Students 
210 Data from Junior Students 

211 Data Collected Prior to Fall Registration 

212 Data Collected One Month Following the First Day of Classes in the Fall 

213 Data Collected One Month Prior to First Semester Final Exams 

214 Data Collected One Month After Spring Semester Classes Begin 

215 Data Collected One Month Prior to Spring Semester Final Exams 

216 Other, for Example, In the Middle of tne Summer Term 

220 Data from Senior Students 

221 Data Collected Prior to Fall Registration 

222 Data Collected One Month Following the First Day of Classes in the Fall 

223 Data Collected One Month Prior to First Semester Final Exams 

224 Data Collected One Month After Spring Semester Classes Begin 

225 Data Collected One Montn Prior to Spring Semester Final Exams 

226 Other, for Example, in the Middle of the SummerTerm 

300 Data from Alumni 

310 Data Collected at Graduation 

320 Data Collected One Year after Graduation 

330 Data Collected Five Years after Graduation 

340 Other, for Example, Data Collected 20 Years after Graduation 

400 Cther, for Example, Data Collected after Students Have "Dropped Out" 

^hese categories and subcategories are based specifically on the data collection experiences of staff in the Office of the Vice Chancellor for Academic 
Affairs at the University of Colorado. Boulder Therefore, they may not be entirely appropriate for other postsecondary institutions 
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PART 1 1 
THE POTENTIAL USES 
OF THE OUTCOMES STRUCTURE 



A number of potential uses have been perceived 
for the Outcomes Structure by various people who 
reviewed it, some of which have been tested in a 
preliminary way and found promising. This sec- 
tion will discuss these proposed uses and provide 
examples. 

Only time and widespread use in a variety of dif- 
ferent institutional and other settings will tell 
whether the so-far promising potential of the 
Structure will "bear fruit." Such use will also 
probably suggest modifications for future ver- 
sions of the Structure, and adaptations in ap- 
plication procedures that need to be made for dif- 
ferent types of users and for different types of in- 
stitutions. The procedures In this document are 
only a start, and feedback from users of the Struc- 
ture is solicited. 

1. Communication 

It is felt that if the terms and their definitions 
in the Structure a r e agreed to by a large num- 
ber of planners and managers in postsecond- 
ary education— and those that are not agreed 
to are modified accordingly in future editions 
of the Structure— it can serve as a useful 
common language where the outcome con- 
cepts and relationships remain constant from 
one use of outcomes information to another. 
In other words, the aim was to develop a 
"workable language" for more effective com- 
munication about postsecondary education 
outcomes. Having kept the terms, definitions, 
and the order of placement as neutral and 
value-free as possible should facilitate such a 
purpose. 

If the Structure terms, definitions, and 
relationships do gain widespread acceptance, 
they should help institutional administrators 
and faculty to communicate more clearly and 
succinctly about the goals and outcomes of 
the institution and its programs to many im- 
portant publics— such as legislators, com- 
munity leaders and citizens, employers, foun- 



dations and other funders, alumni, and 
prospective students and their parents. In this 
"age of accountability," such communication 
is essential in order to maintain public support 
and to attract the needed students. For ex 
ample, in a recent prospective student infor- 
mation needs assessment study coordinated 
by this author, involving questionnaire 
responses by several thousand relevant 
people and interviews with over 500, clear and 
concise information about student outcomes 
ranked very hic,h as information that prospec- 
tive students need. 

The Structure can be just as useful for com- 
munication about outcomes within the in- 
stitution. For example, pairs of members of a 
Student Learning Outcomes Task Force from 
the Council for the Advancement of Small 
Colleges (CASC) visited four small private 
colleges to interview administrators, faculty 
members, and students about the specific 
learning activities emphasized in their 
programs. After discussing activities in each 
interview, categories of the NCHEMS Out- 
comes Structure were introduced in ^n attempt 
to have the interviewees link their activities 
to outcome areas. The Structure was found 
useful for this purpose, but the interviewers 
also noted in talking to faculty from different 
disciplines that the Structure served as a 
"taxonomic device for communicating out- 
comes across disciplinary lines." 

2. Stimulation 

In the preliminary tryout of the Structure, 
there was much evidence that the Structure 
can stimulate administrators and faculty to 
start thinking more about what they are trying 
to accomplish for different groups of students 
at their institution or in their programs, and to 
consider outcomes in a more systematic and 
concrete manner than they have before. For 
example, in the CASC project mentioned 
above, the interviewers had a difficult time 
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switching the conversation from the "comfort- 
able" topic of activities to the "uncomfort- 
able" topic o f outcomes, but the Structure 
categories helped them to do so. As the syn- 
thesizer of the notes from the various inter- 
viewers, Dr. Edward Lundin stated, "Without 
the Outcomes Structure categories, the 
faculty members could have focused on ac- 
tivities without any systematic examination of 
the larger purposes." 

The CASC Task Fo.ce conducted another 
project involving visits to four other small 
colleges, this time jointly with NCHEMS, that 
involved a two-person CASC/NCHEMS team 
going on campus to orient administrators, 
faculty, and students to outcomes concepts 
and the Structure through a half-day 
workshop, and then conducting interviews 
with individuals in each of those groups for 
one and one-half days. In the interviews, 
respondents were first asked to look at the 
audience dimension categories and then to 
point out those they felt were important to 
consider in planning outcomes for the in- 
stitution or for the particular program of 
special concern to them. Then they were asked 
which, if any, student subgroups in their 
program or area of concern had important 
enough outcome needs unique to them that 
they should be considered separately in plan- 
ning for the program or area. 

Next the respondents rated the detailed 
categories of the type-of-outcome dimension 
according to which were considered to be 
essential, which were important, and which 
were less than important as student learning 
outcomes for the important (for planning) 
groups of students in the institution or the 
program of special concern to them. For one 
or two of the categories selected as "essen- 
tial," they were asked to list several specific, 
observable outcomes that were or should be 
emphasized tor those students, and measures 
or indicators that they felt would constitute 
concrete evidence as to whether or not those 
specific outcomes occurred. Then they were 
asked to identify the specific student ex- 
periences and activities that they felt would 
most contribute to the realization of those 
priority learning outcomes that had been iden- 
tified. Finally, they were asked their Im- 



pression of the utility of the Structure now that 
they had gone through this experience. 

All of the students, most of the faculty mem- 
bers outside of education, and many of the 
administrators reported that they had never 
before really thought like this about what their 
institution or program was trying to accom- 
plish for its students, and the majority of these 
people reacted positively to the question 
about utility of the Structure. As one faculty 
member stated (paraphrasing him), "I'm sure 
that I have always had in the back of my mind 
what I specifically wanted to accomplish for 
my students but have never been able to 
state it in formal, concrete, and systematic 
terms like this until now." Another stated, "It 
has made me stop and think about what we are 
trying to accomplish, and it is making the ob- 
jectives in the back of my mind conscious." 
And as a director of development stated 
(paraphrasing again), "I now see many con- 
crete things I never before thought to mention 
to foundations, employers, and others in the 
community that our institution is trying to ac- 
complish for its students." Other people felt 
that the Structure could stimulate people to 
focus on the "why of outcomes," to "raise 
questions," to "think in a concrete manner," 
and to "provide stimulation to guide 
direction." Clearly the Structure would seem 
to have great potential as a device for 
stimulating a variety of people to think more 
about the outcomes they want accomplished, 
and in a more systematic and concrete 
manner than they have previously. This, in turn, 
could lead to useful and stimulating group 
discussion about the outcomes desired for a 
program or institution, and about alternative 
strategies for bringing them about. 



3. Planning, Management, and Evaluation 

There are three "basic processes" that involve 
the direct application of the Outcomes Struc- 
ture. These processes in turn can serve impor- 
tant functions in postsecondary education 
planning, management, and evaluation. Step- 
by-step instructions for these processes are 
provided in Part II of this document. In the 
discussion to follow, the processes will be 
referred to as Process 1, Process 2, and 
Process 3. Their respective descriptions are: 
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through application of the Outcomes 






PROCESS 2 


- Classifying outcomes through application 




of the Outcomes Structure 


PROCESS 3 


- Storing and retrieving outcomes information 




through application of the Outcomes 




Structure. 



Some functions in which these processes may 
be useful are: 

A. Needs Assessment. In this "time of ac- 
countability," the impcrtance of knowing 
the outcome wants and needs of prospec- 
tive clientele groups is increasingly being 
recognized by postsecondary education 
planners and managers. Needs assess- 
ment, as this process is commonly called, 
involves (among other activities) surveying 
these client groups and various concerned 
others to try to ascertain needs, wants, 
and expectancies. The Structure could 
suggest areas of needs assessment focus 
to consider. Furthermore, many of the 
items on such surveys are stated in 
outcome terms, and the Structure provides 
a comprehensive universe of outcome 
categories that can be used to stimulate 
thinking about specific outcomes items 
and to ensure that all areas of potential im- 
portance are considered for coverage in the 
questionnaire. A similar universe of 
categories was found to be useful in 
developing the prospective student infor- 
mation needs assessment questionnaire 
referred to previously. Carefully thought- 
out lists of outcomes and classes of out- 
comes for different audiences resulting 
from either Process 1 or Process 2 can be 
used to determine outcome items for the 
survey forms. 

B. Goal Identification. Another important 
planning function is determining and 
developing priorities for-goals and objec- 
tives of the institution or institutional unit 
of concern. Needs assessment results are 
useful input to such a goal identification 
process, and an application of Processes 1 
and 2 to needs assessment has already 



been mentioned. Some goals (called out- 
come goals) focus on the outcomes that 
are desired and intended for the institution, 
program, or other unit of concern Others, 
which are often related to outcome goals 
and which some people consider also to be 
outcomes because they are a desired 
result of institutional processes, focus on 
such things as levels of resources and par- 
ticular environmental, climate, and process 
conditions or levels {process goals). Goal 
identification involves taking the broad 
mission statements of the institution or the 
missions of institutional units and trans- 
forming them into the more specific goals 
that are appropriate as based on needs 
assessment results. 1 

The "type-of-outcome" dimension of the 
Outcomes Structure is comprehensive in 
its coverage of types of outcomes, is 
hierarchical in nature, and has several 
levels of detail, it has broad outcomes at 
the first level of detail, and increasingly 
specific outcomes at each succeeding 
level of detail. Therefore, since the 
definition of an ''outcome goal" is "desired 
outcome," the Structure is appropriate for 
use in the goal-identification task. (Other 
aids in this process are also available, for 
example, the Educational Testing Service 
Institutional Goals Inventory.) And, in ef- 
fect, that is exactly what Process 1 is all 
about. For each important audience of the 
institution or institutional unit of concern, 
the process leads to a list of desired, 
specific priority outcomes. The inter- 
viewees in the joint CASC/NCHEMS 
project cited earlier were in effect iden- 
tifying their outcome goals for their in- 
stitution or program through an ab- 
breviated version of Process 1, but they did 
not explicitly involve an integration of for- 
mal mission with needs-assessment data. 

C. Goal Assessment. After a list of goals for 
an institution or program has been for- 
mulated, it is desirable to test its adequacy 

1 For a more thorough dlscussk n of the goal Identification process that includes 
a discussion of Integrating mission with needs assessment data and of consensus 
seeking processes concerning goals, see OT Lennlng and SS Mlcek, "Defining 
and Communicating Institutional Miss I on/Role/Scope and Priorities The Needs of 
Different Types of Postsecondary Institutions," paper presented at the annual 
meeting of the American Educational Research Association, San Francisco, April 
1976 (ERIC document ED 121376) 
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uf co/erage. For outcome goals, one way of 
dning this is to use Process 2 to classify all 
cf the goals in the list, and then to apply (or 
reapply) Process 1 to those areas of the 
Structure that seem to be important but to 
which none (or few) of the goals are 
assigned— to determine the "holes" or 
"gaps." 

Staff at the University of Colorado at 
Boulder over a period of years, through 
surveys and interviews of important clien- 
tele groups, had developed several lists of 
intended outcomes that they felt confident 
were comprehensive for their needs. 
Therefore, NCHEMS hired one of the coor- 
dinators of that project to evaluate the 
coverage of those lists using the Outcomes 
Structure in the manner described above. 
To the staff's surprise they found that 
several outcome areas that they con- 
sidered to be quite important had been 
overlooked in the development of their 
lists. Thus, they are revising their lists ac- 
cordingly, and also their freshman student 
questionnaire, which was based in part on 
their lists. 

Spring Arbor College, one of the partici- 
pants in the joint CASC/NCHEMS project, 
used a different method of applying the 
structure to achieve similar results. They 
took lists of outcome goals for the institution 
and separately for each of a number of 
programs— some of the goals had been 
developed through use of the Institutional 
Goals Inventory and others had been 
developed in an open-ended manner— and 
classified them into the five broad (first- 
level of detail) categories of the Outcomes 
Structure Type-of-Outcome dimension. 
Then a goal-profile graph that indicated the 
number of goals assigned to each category 
was drawn separately for the institution 
and each program. For several programs, 
they discovered type-of-outcome categories 
having few goals assigned to them, which 
on reflection they felt should have had 
many goals, while one or more other cate- 
gories ended up with larger percentages of 
the goals than they had anticipated. 



D. Goal Translation. In order to determine the 



measures or indicators on which data 
should be collected for evaluation of the 
institution and/or program, goals must bo 
translated into observable, measurable 
objectives. If Process 1 is carried to its 
most detailed level (specific, concrete, ob- 
servable outcomes), and appropriate in- 
dicators or measures and desired perfor- 
mance levels on them are determined, one 
i.as reached the objective level— which is 
one thing the CASC/NCHEMS project was 
attempting (successfully, considering the 
time available in each interview) to do. Use 
of the sample indicators and measures 
listed in the right-hand column of Appendix 
B may be helpful to stimulate thinking 
about what measures, and what objective 
levels on those measures, would be most 
appropriate for the goals that were select- 
ed. A future project at NCHEMS will 
develop relatively comprehensive lists of 
measures and indicators for selected 
categories of the Structure, from which 
planners and managers can choose those 
most appropriate for them. 

E. Allocation and Utilization Planning and 
Management. One important administrative 
planning function it -esource allocation- 
decisions about how the available money, 
staff, and other resources should be 
applied. Goal identification and translation 
results are important considerations in 
making allocation decisions. So is the time 
when the outcomes are expected to occur 
and how long they are expected to last, 
information that can be determined in 
Process 1 and Process 2. 

Process 1 or Process 2 may also aid in 
other planning decisions, such as program 
activity planning and curriculum develop- 
ment. (When interviewees in the CASC/ 
NCH EMS project were asked to specify the 
most important student experiences and 
activities to use for the different priority 
outcomes they had specified, they were in 
a planning phase.) And that planning in 
turn provides general guidance for day-to- 
day operations. Another planning function 
mentioned under goal translation is what 
measures or indicators to use for collec- 
tion of evaluative data, and performance 
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levels. If certain outcomes are not expect- 
ed but seem to have some probability cf 
occurring, data should also be collected for 
them. A related planning decision is when 
to collect various evaluative data (a 
management function), and the "time" in- 
formation gathered in Process 1 or 2 
should aid this decision. 

In the joint CASC/NCHEMS project inter- 
views, use of the Structure for planning at 
levels within the institution that had not 
been considered when designing the Struc- 
ture were emphasized by some. For ex- 
ample, a number of faculty members felt con- 
fident that use of the Struct, -e could be 
quite helpful in developing courses within 
the curriculum. Furthermore, several 
students and one student personnel ad- 
ministrator expressed the opinion that the 
Structure could be useful to students in 
planning what they want from their college 
careers. 

F. Evaluation. Determining whether the in* 
tended and planned outcomes actually oc- 
curred is an evaluation function, as is the 
determination of whether unintended out- 
comes have occurred. Because it relates 
back to the outcome goals (intended out- 
comes) and the intended quality of in- 
stitutional and program processes for 
achieving those goals, success in the 
evaluation phase depends greatly on the 
goal-setting and planning phases. The data 
for the measures or indicators selected 
during the planning process, with the help 
of Process 1 or Process 2 and Appendix B, 
have presumably been collected at ap- 
propriate times (times also decided during 
the planning process through input 
stimulated by the Structure "time" dimen- 
sion). If so, the actual outcomes and out- 
come levels can be compared to the 
desired outcomes and outcome levels, and 
any unintended outcomes that are observed 
may be assessed. Then the evaluation 
results and interpretations can be fed back 
to guide new goal setting and planning. If 
data are being collected for a large number 
of measures pertaining to thousands of 
students at several or many different times 
during their college careers, computerized 



storage of the outcomes information, using 
Process 3, becomes essential. Inter- 
pretations of the discrepancies found be- 
tween each important desired outcome and 
its associated actual outcome can be fed 
back to the planning process to aid in 
decisions about modification, revision, 
replacement, or termination of activities, 
resources, direction of effort, and so forth. 

The evaluation phase involves much more 
than just comparing actual outcomes to in- 
tended outcomes. It includes data and 
analyses (plans for which should also have 
been a part of the planning phase) to an- 
swer questions such as the following: 
Does the outcome result from this post- 
secondary education institution or in- 
stitutional unit? How do we know? Have 
resources, including those embodied in the 
entering students, been accounted for? 
What is the value of the outcomes as com- 
pared to their cost? 

Evaluation is closely related to the function 
mentioned earlier of communicating with 
funders and other concerned publics. (The 
image of the institution and its programs is 
crucial in attracting funds, students who 
will be happy there, and a goo < staff.) It is 
hoped that more precise evaluative data 
about the institution and its programs will 
be available from application of the Struc- 
ture. With this knowledge, and more 
precise terminology, administrators and 
faculty may be able iO communicate more 
succinctly what their institution is accom- 
plishing for individuals and society, and 
how their institution and its programs dif- 
fer from their counterparts. 

4. Research 

All three of the ''basic processes" outlined in 
the preceding subsection may be useful in 
both basic and applied outcomes research. 
Process 1 results can suggest gaps in the 
comprehensive set of outcomes that need to 
be researched, either for specific types of out- 
comes or large areas of outcomes. It can also, 
when considered with the sample measures 
and indicators presented in Appendix B, 
suggest outcomes and classes of outcomes 
where the development of new measures 
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should be especially focused. Results of 
studies found in the literature, pools of out- 
comes questionnaire items used in various 
studies, pools of outcome measures that have 
been collected, literature references of out- 
come studies, data collected in large-scale 
studies, and so forth, can be classified using 
Process 2. They they can be stored in a com- 
puter storage bank within locations keyed to 
the most detailed classes, and be made 
available for retrieval in aggregated or 
disaggregated form at the proper time for any 
analyses desired (Process 3). In addition to the 
three basic processes, the relationships 
among the various classes and subclasses of 
the Structure itself can provide research 
hypotheses that should be tested out. One 
reviewer of the Structure reported that his 
examination suggested to him an extensive 
outcomes research agenda that needs to be 
commenced. In addition to the research 
questions implied above, other possible 



questions stimulated by the Structure include 
the following: What are the real (as opposed to 
folklore) outcomes of postsecondary educa- 
tion? What outcomes naturally go together- 
form clusters or constellations? Which 
outcomes last longest? Which clusters of 
outcomes are most valuable? What indicators 
and measures are most appropriate for the 
different categories of the Structure, and how 
can we go beyond opinions to observing facts 
and "hard data?" Which processes yield 
which outcomes for whom? What are the 
relationships of costs to value added for dif- 
ferent types of outcomes? What factors im- 
pact on outcomes of different types, and how 
can we understand such relationships? How 
do the outcomes of institutions, programs, 
and other postsecondary education settings 
vary for different audiences? Which outputs 
(direct outcomes) lead to which impacts (in- 
direct outcomes)? 
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PART 1 1 1 
GUIDELINES FOR USING THE 
OUTCOMES STRUCTURE TO ACCOMPLISH 
THREE BASIC PROCESSES 



The Outcomes Structure is a framework for use in 
organizing and classifying information about 
educational outcomes. It allows placement of 
outcomes along three basic dimensions: audi- 
ence, type-of-outcome, and time. Unless one has 
decided who or what is to receive or be affected 
by the outcome of interest— the audience— direc- 
tions for a course of action are missing. The same 
is true concerning whether we are striving to 
maintain or to change characteristics and con- 
ditions, and specifically what it is we are trying to 
maintain or change— the type-of-outcome. 
Similarly, planning related to educational out- 
comes can be adversely affected if we give no 
thought to when the outcomes should occur. And 
evaluation may not be useful if evaluative data are 
not collected at the proper time. 

This section describes three basic processes for 
use of the Structure: (1) generating useful lists of 
desired or expected priority outcomes for an in- 
stitution or institutional unit, (2) classifying the 
outcomes or outcomes information in available 
lists, for an institution or institutional unit, and 
discovering where there are important "holes" or 
"gaps," and (3) storing and retrieving outcomes 
information. As discussed in Part II, these 
processes can be applied in various ways to aid 
important decision processes in the areas of 
needs assessment, goal identification, goal 
assessment, goal translation, planning and 
management, and evaluation. Furthermore, they 
can be applied at the institution-wide level, at the 
program or departmental level, and at other levels 
such as planning courses, student planning, state 
system planning. 

Process 1: Generating Lists of Priority Outcomes 

Developing a listing of the desired outcomes is 
especially important for planning, whether it be 
planning for a course, planning for a curricular 
program, planning for an academic department, 



planning for an institution, or planning for a 
system of institutions. (As mentioned earlier, it 
also is extremely important for tasks such as 
evaluation, but these tasks derive and extend 
from the planning function.) In addition, the 
listing should give some indication of priority: for 
example, which of the outcomes are essential, 
which are valuable, and which have little or no 
value. Process 1 consists of applying the Out- 
comes Structure to develop such a listing. 
Decisions (choices) will be made at each of the 
steps that follow. "Who decides 0 will not be 
discussed here— it depends on local factors such 
as organizational arrangement, administrative 
styles, and customary procedures. It Is strongly 
recommended, however, that some person have 
clear authority and assigned responsibility for 
making these decisions and that well-thought-out 
decision procedures be developed. In addition, it 
is important that representatives of each major 
concerned group on campus, including students, 
be involved through providing input for these 
decisions. There are a number of ways this can be 
done— for example, "town hall" meetings, com- 
mittees, interviews, surveys, card sorts, or the 
Delphi technique may be used. It also would be 
desirable to get input from concerned off-campus 
groups such as community leaders, alumni, and 
members of the community at large. Although the 
user may wish to modify the procedures from 
those that are suggested, the following are 
general procedures that evolved from preliminary 
tests of the Structure. But let it be made clear, 
local options are encouraged, whether they be 
modifications in .ne procedures for using the 
Structure or in the content of the Structure itself. 
For example, in the joint CASC/NCHEMS visits to 
colleges, time was short so some of the steps 
were omitted and all others abbreviated. 

Outcomes for Whom? Before beginning to think 
about the types of outcomes or about specific 
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outcomes, it is important to consider the audi- 
ence that is meant to receive benefits from or in 
other ways be affected by the academic in- 
stitution or the unit of concern within that in- 
stitution. There are many potential audiences (for 
example, students and those who influence 
students, such as parents and counselors; gover- 
nors, legislators, mayors, and voters; and society, 
as represented by employers and buyers [that is, 
consumers] in the market place). The purpose is 
to choose those which should be of concern in 
planning for outcomes. Too often in institutional 
planning, the target audiences for major intended 
outcomes are not formally specified, which 
reduces the potential for tailoring the planned in- 
stitutional activities to fit the characteristics and 
situation of the audience. The "audience" dimen- 
sion of the Structure was developed \o facilitate 
determining the people, the communities, or the 
other entities specifically intended by the in- 
stitutional or program planners to receive or be af- 
fected by outcomes of the institution/institutional 
unit. The primary categories of the audience 
dimension are shown in Table 1 on page 4. (As 
mentioned previously, cod^ numbers have been 
assigned to the categories, and the coding 
scheme is discussed in Appendix A.) Note that 
there are five major (first-level) audience 
categories, one being an "other" category. Not 
only do "other" categories make the Structure 
more flexible and able to meet the needs of 
unique institutions, but also they stimulate the 
user of the Structure to ask whether there are ad- 
ditional audiences for the institution's outcomes 
that should be considered. 

STEP1 Decide which of the broad audience 
categories in Table 1 are important 
recipients of the outcomes of the in- 
stitution or institutional unit of concern. 

STEP 2 Look at Table 2 on page 5. For each 
major audience category selected in 
Step 1, decide which of the sub- 
categories (second-level) in Table 2 are 
important recipients of the outcomes of 
the institution or institutional unit of 
concern. 

STEP 3 It may be helpful when thinking about 
the outcomes to be even more specific 
than the categories listed in Table 2, 



concerning who or what are the audie - 
ces of concern. Therefore, for each 
audience subcategory selected for 
focus, it is suggested that specific 
significant groups within the category 
be listed. Examples for the "students" 
category in Table 2 might be "students 
who entered out of high school" and 
"older students." (Instead of older 
students it might be desirable to have 
even more specific groups, such as 
"veterans," "homemakers," "retrainees," 
and "retired persons.") A set of examples 
for students that might be especially 
pertinent for a curricular program are 
"students majoring in the program" and 
"nonmajors taking courses in the 
program." Examples of more detailed 
categories for private enterprise com- 
munities might be "major corpora- 
tions in the community" (it could be 
helpful to list them if there are only a few 
or to list the different types), "small 
business firms in the community ' (again 
a listing of different types could be 
useful later), and "area farmers." Note in 
the last set of examples that we in effect 
combined two quite different second- 
order categories: "local community" and 
"private enterprise communities." 

The lists of more specific audience 
types generated for the various 
"audience" subcategories c ? concern in 
Table 2 can be used in several ways, 
whichever seems more helpful to the 
person using the Structure: (a) they can 
be used as cues to stimulate thinking 
about specific outcomes for the Table 2 
categories selected, (b) they can be 
made into third-level categories of the 
audience dimension to insure more 
specificity in categorizing outcomes, 
and (c) some of them can be handled as 
in (a) above, and others as in (b) above. 

STEP 4 The audience categories on which you 
wish to focus specific attention in 
developing lists of desired educational 
outcomes have been determined in 
Steps 1-3. Now print each of these 
category names at the top of separate 
sheets of blank paper (one audience 
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category per sheet). In addition, for each 
category, print near the top of the sheet 
any audience examples that would 
presumably be helpful in stimulating 
thinking about specific desired out- 
comes for that audience category (as 
suggested in a of Step 3). 

What Outcomes? Now that you have a group of 
sheets with an important audience category 
printed at the top of each, you are ready to proceed 
to the "type-of-outcome" dimension of the Struc- 
ture. The purpose now is to determine which out- 
comes are important for each audience that has 
been chosen as sufficiently important to m 
separate consideration in planning for the in- 
stitution or program of concern. The type-of- 
outcome dimension identifies whether or not 'he 
outcome involves a change in status (mainte- 
nance versus change), and the basic entity that is 
maintained or changed (such as knowledge and 
understanding, skills and competencies, at- 
titudes and values, certification status, income, 
standard of living, social interactions, and art 
forms or expression). 

STEP5 Order the sheets according to which 
audiences should be focused on first in 
listing specific outcomes, and which 
last. That way, if you run out of time, you 
will have completed lists of outcomes 
for what you consider to be the most im- 
portant or relevant audiences for the in- 
stitution/institutional unit of concern. 
Criteria that may be helpful in this or- 
dering process include such factors as 
whether the primary impact is likely to 
be direct or indirect (the institution often 
has more control and opportunity to af- 
fect direct outcomes as compared to in- 
direct outcomes), relative numbers of 
outcomes expected for each group, how 
basic and fundamental the outcomes for 
each group seem to be, and the per- 
ceived importance of each group relative 
to the others. 

Some people will prefer to focus on the 
outcomes for only one audience at a 
time. Others will find it useful and ef- 
ficient to focus on the outcomes for 
more than one audience at a time. It is 
extremely difficult to focus on outcomes 



separately for more than two or three 
audirnces at a time, however, based on 
experiences in the CASC/NCHEMS joint 
project. 

STEP 6 The five major (first-level) categories of 
the type-of-outcome dimension, and 
definitions for each, are shown in Table 
3 on page 6. Across each of the "audi- 
ence sheets" selected in Step 5 list the 
type-of-outcome categories from Table 3 
that you consider important for that 
audience group. To illustrate, this step 
is shown in Figure 1 on page 18 for a 
political science department curriculum 
committee at Alpha College (a fictitious 
college). They had decided to limit their 
focus to two audience groups, "students 
majoring in political science 1 ' and 
"other students taking political science 
courses," and thus they had two 
audience sheets on which they were 
working. 

STEP 7 Table 4 on page 7 lists the second- and 
third-level categories for each first-level 
category of the type-of-outcome dimen- 
sion, and definitions for all of these 
subcategories are given in Appendix B. 
Now go to the first audience sheet and 
do the following: 

a. For each first level "type" category 
listed on the audience sheet, jot 
down on a scratch pad all second- 
level categories that your analyses 
(whether they be subjective or objec- 
tive analyses) indicate to be impor- 
tant for that audience (referring to the 
definitions of Appendix B as needed). 2 

b. For each second-level category 
decided on in a, list on the audience 
sheet all third-level categories that 
you consider to be important 
(referring to the definitions of Appen- 
dix Bas needed). 

Repeat a and b for every sheet. Figure 2 
on page 19 shows the Alpha College 
Political Science Committee sheets after 
Step 7 was completed. 

'if you are unsure about whether a category should be Included, Hat it Don't 
waste time evaluating further at this level because when you get down to the sub- 
categories for that category it will probably become clear whether or not it should 
be included In the CASC/NCHEMS test of the Structure, this was a major reason for 
the process being slowed 
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Figure 2 

THE AUDIENCE SHEETS OF THE POLITICAL SCIENCE CURRICULUM COMMITTEE AT ALPHA 
COLLEGE AFTER THEY HAD COMPLETED S7FP 7 



Students Ma/oringin Political Science Sheet 1 
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Figure 2 
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STEP 8 Traditionally, maintenance and change 
have been considered by many to be 
fundamental purposes of postsecondary 
education (for example, preserving the 
culture and improving the condition of 
mankind). Too of'en planners focus only 
on bringing about change, and do not 
consider maintenance outcomes in their 
planning. For example, in curriculum 
development for advanced courses, it 
may be important to build in exercises at 
periodic intervals that will keep the 
students from getting "rusty" on impor- 
tant basic skills that would not other- 
wise be applied in the advanced 
program. For these reasons, and because 
these two types of outcome goals 
require quite different orientations and 
approaches in planning and manage- 
ment, they were made fourth-level 
categories of the type-of-outcome 
dimension. Definitions for each 
category follow: 

1. Maintenance— Outcomes that result 
in keeping the status quo; in 
stabilization, reproduction, or preser- 
vation. Examples include preserving 
cultural values, restoring artifacts 
and paintings, keeping up the 
educational level of the family, and 
skill maintenance provided by in- 
service education. 

2. Change— Outcomes that result in 
alteration of the status quo, in 
reorganization, modification, revision 
(improvement or otherwise), or 
replacement. Included are economic 
and social mobility, degree or cer- 
tification awarded the student, in- 
creased knowledge and skill level, 
new art forms, technological in- 
novations, medical discoveries, and 
so forth. 



For all of the type-of-outcome categories 
listed on each audience sheet, indicate 
in some way (for example, by the letters 
M, C, and B, or by a fourth-digit code of 
1, 2, or 0) whether the category should 
emphasize only maintenance, only 
change, or both maintenance, and 
change. In the case of the Alpha College 



Political Science Committee, they 
decided to focus on change in each 
case. Therefore, they changed the fourth 
digit of each outcome code number from 
'0' (both) to '2' (change). 

9 For some purposes, such as curriculum 
development and planning for a course, 
still more detail may be needed than the 
third-level categories of Table 4. Logic or 
specialized taxonomies that are avail- 
able may suggest the need for a set of 
fifth-level categories for some of the 
type-of-outcome categories listed on the 
sheet. For example, using Bloom's 
Taxonomy of Educational Objectives 
(1956), intellectual skills (code number 
2250) can be divided into: "ability to 
comprehend," "ability to apply," "ability 
to analyze," "ability to synthesize," and 
"ability to evaluate." Similarly, physical 
and motor skill outcomes (code number 
2280) could be divided into more 
detailed categories using one of the six 
psychomotor taxonomies listed in Len- 
ning (1977a). As a final example, "at- 
titudes and values" (code number 2310) 
could be divided into the following 
categories listed in the NCHEMS Inven- 
tory of Higher Education Outcome 
Variables and Measures (Micek and 
Wallhaus, 1973): intellectual disposi- 
tion, political attitudes and values, 
racial/ethnic attitudes ard values, 
personal ethics, social conscience, 
religious and spiritual attitudes and 
values, and career attitudes and values. 



If you determine that more detailed 
categories are needed for any of the 
third-level categories listed on your 
audience sheets, decide what those 
categories should be— either through 
application of an available taxonomy or 
through open-ended logic. Then replace 
those listed categories with the even 
finer categories you have selected. 
Figure 3 shows the Alpha College 
Political Science Committee sheets af- 
ter Step 9 was completed. Note that they 
replaced "intellectual skills" with fifth- 
level categories taken from Bloom's 
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Figure 3 



THE AUDIENCE SHEETS OF THE POLITICAL SCIENCE CURRICULUM COMMITTEE AT ALPHA 
COLLEGE AFTER THEY HAD COMPLETED STEP 9 
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Taxonomy, and added a fifth digit to the 
code number for each of those fifth-level 
categories. 

STEP 10 For each detailed "type-of-outcome" 
category on each of the audience 
sheets, on separate sheets of paper you 
are now ready to list important specific 
outcomes you might expect (not just 
those you desire) of that type for that 
audience. Some users of the Structure 
may want to develop long lists of ex- 
tremely detailed outcomes for each 
category, while others may want to 
develop shorter lists of outcomes that 
are less specific. Several suggestions 
might help stimulate your thinking as 
you attempt to identify specific out- 
comes for each category: 

a. Think in terms of outcomes you 
would consider to be: (1) "positive in 
value," (2) "neutral in value/' and 
(3) "negative in value." For example, 
many might consider "increased 
ability to analyze problems" to have 
positive value and "increased drug 
use" to have negative value. 

b. Think in terms of outcomes that are: 
(1) easy to measure, and (2) difficult 
to measure. 

c. Think in terms of outcomes that are: 
(1) products— tangible, concrete en- 
tities that endure with time, such as 
knowledge, skills, a program com- 
pleter, a degree, a job, a book 
published; (2) events— observable, 
tangible transactions or sets of 
behaviors that do not endure with 
time, such as a seminar, a concert, a' 
graduation exercise, be s ng listed in 
Who's Who in America; an* (3) con- 
ditions— intangible but real circum- 
stances, such as morale, satisfac- 
tion, an attitude or belief, an ap- 
preciation, social equality, achieve- 
ment, orelimination of polio. 

d. Think in terms of outcomes that are: 
(1) outputs— the direct end products, 
events, or conditions that are expect- 
ed to result from the application of 
the institution or institutional unit 



processes to transform the various 
inputs, such as achievement levels, 
specialization of knowledge, degrees, 
program completers, publications, 
cultural or entertainment events; and 
(2) impacts — the consequences 
of outputs and earlier impacts 
(that is the indirect products, events, 
or conditions produced), the dif- 
ferences they make, such as a pro- 
gram completer's ability to obtain 
and hold a job, the security and In- 
come or the prestige the job gives a 
person, the increased gross national 
product that results from increased 
income of individuals, the increased 
standard of living and quality of life 
that may be associated with in- 
creased gross national product, and 
soon. 

The outcomes listed may be quite 
broad— like those examples listed 
above— or they may be very specific, 
such as the percentage scoring above a 
particular level on a certain test or inven- 
tory. (Examples of specific measures 
that could be used for each detailed 
category of the type-of-outcome dimen- 
sion are given in the right-hand column 
of the pagas in Appendix B.) How 
specific they should be depends on the 
philosophy, needs, and desires of the 
person using the Structure. 

STEP 11 Now that you have a list of specific 
desired outcomes for each important 
audience, it will be useful to separate all 
the outcomes listed into priority groups 
(and to record the priority status of each) 
such as the following: (1) very important 
outcomes and (2) important outcomes. 
You may desire to use other categories 
than these. As with the other steps, it 
will be beneficial toget input from major 
concerned groups of people, both on 
and off the campus. 



To illustrate what the lists coming out of Step 11 
might look like, Figure 4 shows the first page of 
intended outcomes for political science majors 
developed by the Alpha College Political Science 
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Figure 4 



THE FIRST PAGE OF THE OUTCOME LIST FOR STUDENTS MAJORING IN 
POLITICAL SCIENCE THAT HAD BEEN DEVELOPED BY THE POLITICAL SCIENCE 
CURRICULUM COMMITTEE AT ALPHA COLLEGE ATTHE COMPLETION OF STEP 11 



INTERESTS: 



*1 Increased student interest in the theory, principles, and practices of government and politics 

*2 Increased student interest in learning about different political concepts, philosophies, and ideologies 

3 Increased student interest In entering politics, government, or public administration 

4 Increased student interest in international relations and world views 

5 Increased student interest in the analysis and solution of social problems 
*6 Increased student interest in constitutional law and jurisprudence 

*7. Increased student interest in the dynamics of change in American society 

*8 Increased student interest in the public economy and its operation. 
DESIRES, AIMS, OR GOALS. 

1 Increased student desire and intention to prepare for advanced and specialized study in political science. 

2. Development by students of plans for political science careers in teaching, government, business, and law 

3. Development in students of a desire for public service 

*4. Development in students of a desire for conducting political analysis and action while they afe still students 
on campus. 

*5. Development in students of an aim to be a model in the conduct of citizenship responsibilities. 

6. Development in students of an aim to uphold democracy as ou political system. 

*7. Development in students of a desire to improve the character of the relationships between citizens and 
government, and of the values and criteria by which quality of political life is judged 

*8. Development in students of intentions to uphold citizen rights and liberties, and the separation of church 
and state. 

CITIZENSHIP A ND FA MIL Y MEMBERSHIP SKIL LS 

*1. Increased student ability to intelligently conduct citizenship responsibilities. 

*2 Increased student ability to understand the basic political processes and institutions as they operate m different 
national and cultural contexts. 

*3. Increased student ability to critically analyze public opinion and propaganda. 

*4. Increased student ability to analyze voting patterns, political participation, techniques of political action, 
legislative or governmental actions, and governmental bureaucracy. 

*5. Increased student ability to analyze and solve social problems. 

*6. Increased student ability to recognize the distinction between what public figures may say and their motivations 
for saying it. 

*7. Increased student ability to deal with the moral dilemma in politics between principle and compromise 
CREATIVITY SKILLS 

1 Increased student ability to run an innovative, creative political campaign. 
•Outcomes considered "essential", the others are considered Important 
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Curriculum Committee as it looked when it 
came out of Step 11. Note that they put an 
asterisk by each outcome they considered 
"essential," and the others were considered 
••important." 

When Will the Outcomes Occur? Time has 
always been considered an especially important 
factor fo. many student outcomes. Particular 
teaching content and activities do not have much 
effect until the time a student is "ready" for them. 
Presumably there are times when curriculum 
materials and emphases of different types will 
have maximum influence. Thus, prerequisite ex- 
periences and/or courses must often be specified 
for different advanced courses within a curricu- 
lum. Another example is that many students may 
not be ready to choose a major until they have 
chosen a vocation. 

Outcomes that occur at or prior to graduation are 
more often a direct focus of planning than are im- 
pacts that occur after graduation, because the in- 
stitution has some control over them. On the 
other hand, if the relationship of the earlier out- 
comes to the postgraduate impacts can be ascer- 
tained, the later impacts may have some influ- 
ence on priorities. Furthermore, communications 
to the general citizenry about outcomes often 
must have a primary focus on postgraduate im- 
pacts—whether they are actual or presumed— 
because postgraduate outcomes are of special 
concern to the public. 

Another "time" consideration is how long the 
outcome will endure; for example, is it of long 
duration or of short duration? Time is also impor- 
tant for planning and management related to non- 
student outcomes, such as research and scholar- 
ship outcomes, art forms and works outcomes, 
and economic outcomes for society. Because of 
its critical importance to postsecondary educa- 
tion planning and management, "time" was made 
a dimension of the Outcomes Structure. No 
categories were assigned to this dimension, 
however, because (1) agreement could not be 
reached on such categories, (2) time categories 
needed vary with the audience and type-of- 
outcome, and (3) the time categories needed by an 
institutional official will vary according to 
philosophy and use context. For example, a 
curriculum developer will undoubtedly need a 
quite different set of time categories than will a 



campus administrator concerned with general 
campus-wide outcomes. It was decided to leave 
any specific categorizing (and definitions for 
those categories) up to each individual using the 
Outcomes Structure. 

STEP 12 Decide on the categories and sub- 
categories you need for the "time 
dimension." 

STEP 13 Next to each of the outcome statements 
in the lists coming out of Step 11, place 
the names (code numbers or letters 
would be more efficient) of all "time" 
categories that apply to that outcome. 
You are now ready to analyze and to use 
the listings of outcomes (one list for 
each of your important audiences). 

See Figure 5 for an illustration of what the final 
lists— coming out of Step i 3— might look like. 
(Note that the time differentiation in the 
illustration was whether the outcome should be 
emphasized in upper-division courses [U], lower- 
division courses [L], or both upper and lower 
division courses [B].) Keep in mind, however, that 
this is merely the way a political science curricu- 
lum committee at Alpha College designed it. 
You might very well wish to organize your lists 
differently. 



Process 2: Classifying Outcomes 

Some institutions and institutional units have 
periodically developed lists of desired outcomes 
or outcome goals and objectives. Taking such a 
list of outcomes and classifying it through ap- 
plication of the Outcomes Structure can provide 
information to institutional officials that will 
assist them as they evaluate the breadth as well 
as the adequacy of the list. Such a procedure may 
confirm and stimulate the realization that the 
desired outcomes are concentrated in only a few 
specific areas, that some are too broad and 
general, that some are stated in unduly vague 
terms, that some need to be translated into more 
specific outcomes (and the Structure can help do 
this), that few, if any, of the outcomes are in cer- 
tain areas of supposed concern to the institution 
or institutional unit, that certain outcomes need 
to be added to the list, and so forth. 



ERIC 



24 



33 



Figure 5 

THE FIRST PAGE OF THE OUTCOME LIST FOR STUDENTS MAJORING IN POLITICAL 
SCIENCE THAT HAD BEEN DEVELOPED BY THE POLITICAL SCIENCE CURRICULUM 
COMMITTEE AT ALPHA COLLEGE AT THE COMPLETION OF THE FINAL STEP, STEP 13 

INTERESTS' 

B * 1 Increased student interest in the theory, principles, and practices of government and politics 
B * 2 Increased student interest in learning about different political concepts, philosophies, and ideologies. 
U 3 Increased student interest in entering politics, government, or public administration. 
U 4. Increased student interest in international relations and world views. 
U 5 Increased student interest in the analysis and solution of social problems. 
U * 6 Increased student interest in constitutional law and jurisprudence 
L * 7 Increased student interest-in the dynamics of change in American society 
U* 8 Increased student interest in the public economy and Its operation. 
DESIRES, AIMS, OR GOALS' 

U 1 Increased student desire and intention to prepare for advanced and specialized study in political science. 
U 2 Development by students of plans for political science careers in teaching, government, business, and law 
B 3 Development in students of a desire for public service. 

B * 4 Development in students of a desire for conducting political analysis and action while they are still 
students on campus. 

B * 5 Development in students of an aim to be a model in the conduct of citzenship responsibilities. 

L 6 Development in students of an aim to uphold democracy as our political system. 

B * 7 Development in students of a desire to improve the character of the relationships between citizens and 
government, and of the values and criteria by which quality of political life is judged. 

L * 8 Development in students of intentions to uphold citizen rights and liberties, and the separation of church 
and state 

CITIZENSHIP AND FAMIL Y MEMBERSHIP SKILLS' 
L * 1 Increased student ability to intelligently conduct citizenship responsibilities 

B * 2. Increased student ability to understand the basic political processes and institutions as they operate in 
different national and cultural contexts. 

B * 3 Increased student ability to critically analyze public opinion and propaganda. 

U * 4 Increased student ability to analyze voting patterns, political participation, techniques of political action, 
legislative or governmental actions, and governmental bureaucracy. 

B* 5. Increased student ability to analyze and solve social problems 

L * 6. Increased student ability to recognize the distinction between what public figures may say and their 
motivations for saying it. 

U * 7. Increased student ability to deal with the moral dilemma in politics between principle and compromise. 
CREATIVITY SKILLS 

U 1 . Increased student ability to run an innovative, creative political campaign 



* Outcomes considered "essential", the others are considered "important " 

KEY 



L - Lower Division Courses 
U = Upper Division Courses 
8 = Both Upper & Lower Division Courses 
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Although o.ie person can classify the outcomes in 
a list, it is suggested that several persons be in- 
volved. Included should be representatives of 
major groups having effective control or a 
legitimate concern about what the educational 
outcomes of the institution/institutional unit 
should be (for example, administrators, faculty, 
students, and graduates). 

Rather than write down by each outcome item the 
name of the Structure category or categories to 
which it is assigned, it will be much more efficient 
and workable to use the code numbers that have 
been assigned to all categories of the Structure. 
As indicated earlier, each category of each 
dimension of the Structure has a code number 
assigned to it, and by looking at the code number 
one can deduce the level of detail of the category 
that has been assigned. (Again see Appendix A 
for a discussion of the coding scheme for the 
Structure.) Since few, if any, currently available 
lists of institution-wide or institutional-unit out- 
comes reference the time the outcome is expect- 
ed to occur or give differential indications of who 
is to receive or be affected by the outcome, the 
discussion in this section will focus on 
classifying outcomes into categories of the type- 
of-outcome dimension. If the outcome statements 
on your list do specify the time when the outcome 
is expected or intended to occur or be empha- 
sized, and the audience or audiences who are 
to receive that outcome, use the same procedures 
to classify on those dimensions as are outlined 
below for the type-of-outcome dimension. 

The procedures presented below are not limited 
to use in classifying lists of outcomes and out- 
come goals or objectives. They can also be used 
to classify, in terms of outcomes, such things as 
research findings in the literature, questionnaire 
items, institutional and program activities aimed 
at outcomes, needs, campus environments, and 
philosophical orientations. 



STEP1 Note in Table 3 on page 6 that the first 
digit of the code number identifies the 
first-level categories— 1 =economic 
outcomes, 2 = human characteristic 
outcomes, 3 = knowledge, technology, 
and art form outcomes, and so forth. 
(The zeros in the succeeding digits of 
each code number indicate that the out- 
come is not being classified at the more 



refined levels of detail that are 
available.) In front of each outcome 
statement on your list, write "1" if it is 
an economic outcome, "2" if it is a 
human characteristic outcome, and so 
on, until all statements in the list are 
classified at this level of detail. (In each 
case, leave enough space after the digit 
so you can add additional digits.) 

STEP 2 Table 4 on page 7 shows the second- 
and third-level categories applying to 
each first-level category of the type-of- 
outcome dimension. For each outcome 
statement on the list that has a "1" in 
front of it, determine whether the second 
digit should be "1," "2," "3," or "4" by 
examining the title of each of the four 
second-level categories for that major 
type of outcome (1 = economic access 
and independence, 2 = economic 
resources and costs, 3 = economic 
production, and 4 = other economic out- 
comes). Each of the second-level 
categories is defined in Appendix B 
(page 39) and you may need to refer to 
those definitions in determining which 
second-level category specific econom- 
ic outcomes should be assigned to. 
Some of the economic outcome state- 
ments may be too broadly stated to 
be classified in any second-level 
economic outcome category. If they 
cannot be classified at this level, assign 
them a second digit of zero. If such an 
outcome statement applies to two of the 
categories, you can either assign it two 
second digits (which will give it two dif- 
ferent code numbers) or give it a zero, 
whichever you prefer. 

Next, for all outcomes in the list that 
have a "2" in front of them, determine 
which of the nine second-level 
categories of "human characteristic 
outcomes" apply. Assign these out- 
comes a second digit number in the 
same manner that you did for economic 
outcomes. If it cannot be classified at 
this level, give it a second digit of zero; if 
it is an aspiration outcome, give it a 
second digit of "1"; if it is a competence 
and skill outcome, give it a second digit 
of "2";andso forth. 

3d 
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In the same manner as you assigned a 
second digit code to the economic and 
the human characteristic outcomes on 
the list, assign second-digit codes to 
outcomes assigned to the three other 
first-level categories. When this is done 
you are ready to classify at the third 
level of detail, assuming that you desire 
to classify at that level of detail. For 
some lists (such as broad overall in- 
stitutional goals), classifying to the 
second-level of detail may suffice. 

STEP 3 Give a third digit of zero to each out- 
come statement in the list that has a 
second digit of zero. 

STEP 4 For each outcome statement in the list 
that has a "11" in front of it (economic 
access and independence outcomes), 
determine which of the three third-level 
categories listed in Table 3 (for that 
second-level category) applies, and 
assign it a third digit accordingly (for 
example, 111 = economic access and 
1 13 = income and standard of living). As 
with the second-level categories, each 
third-level category is defined in Appen- 
dix B. You may need to look at these 
definitions to classify at this level of 
detail. If the outcome cannot be 
classified at this level of detail, assign it 
a third digit of zero. 

In like manner, assign third digits to the 
codes for all outcomes on the list that 
have "12" in front of them, then to all 
outcomes that have "13" in front of 
them, then to all outcomes that have 
"14" in front of them, then to all out- 
comes that have "21" in front of them, 
then to all outcomes that have "22" in 
front of them, and so forth. Once all of 
the outcomes on the list have been 
assigned third-digit codes, you are ready 
to assign fourth-digit codes (once again, 
assuming that you desire to classify at 
that level). 

STEP5 If you do not desire to classify at the 
fourth level of detail, give every outcome 
statement on the list a fourth digit of 
zerc If you do desire to classify at the 



fourth level of detail, start with the first 
outcome statement on the list and 
proceed one-by-one to the last item on 
the list, assigning fourth digit codes as 
follows: 

1— The outcome statement indicates 
that it is a "maintenance" outcome, 
an outcome that will result in keeping 
the status quo, in stabilization, in 
reproduction, or in preservation. 

2— The outcome statement indicates 
that it is a "change" outcome, an 
outcome that results in alteration of 
the status quo, in reorganization, in 
modification, in revision, or in 
replacement. 

0— The outcome statement: (a) indicates 
that it is including both maintenance 
and change, or (b) gives no indication 
as to whether it is referring to main- 
tenance or to change. 

For an example of a case where these procedures 
were actually applied, see Appendix C. All of the 
outcomes in the NCHEMS Inventory of Higher 
Education Outcome Variables and Measures were 
listed on a sheet. Then one or more four-digit code 
numbers were assigned to each using the 
NCHEMS Outcomes Structure and going through 
the five steps that have been described in this 
section. Note that the procedures were modified 
slightly, instead of placing the Structure code 
numbers in front of the outcomes on the list (as 
specified in Step 1), they were placed behind the 
outcome statements to facilitate including the 
descriptive "type-of-outcome" name for each 
Structure category assigned. Modifications of the 
procedures outlined here (or of the content of the 
Structure itself), to meet special local needs or 
situations, are encouraged. 

Appendix C has another use in addition to 
illustrating the application of the five steps out- 
lined above. Those who have in the past applied 
the NCHEMS Inventory to institutional planning 
functions can use Appendix C to transform their 
previous work into Outcomes Structure terms, if 
they desire to do so. 

After examining the distribution of code numbers, 
you may decide that too many outcomes on the 
list are not specific enough for your needs, or that 
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important outcome areas are not emphasized 
enough or are completely ignored by the list. If 
this is the case, you may wish to expand and in 
other ways modify the outcomes list. Such ex- 
pansion and other modifications can be accom- 
plished by following the procedures outlined for 
Process 1 on pages 15-24 

Process 3: Storing and Retrieving Outcomes 
Information 

The statement was made in the introduction to 
this document that "having a wide variety of out- 
comes information without any structure is 
analogous to possessing a file cabinet in which 
the contents are arranged randomly." Outcome 
information to be stored could consist of a pool .' 
thousands of behavioral objectives that have 
been collected from different curriculum 
developers, a pool of outcome questionnaire 
items that have been collected from different 
developers of outcomes instruments, a pool of 
different measures that have been developed for 
different outcome variables, a collection of 
references from the literature of postsecondary 
education outcomes, and so on. They could be 
stored in a filing cabinet, in a microfiche or 
microfilm information storage and retrieval 
system, in a computerized information storage 
and retrieval system, or in other ways. 

In most cases of applying the outcomes Structure 
to develop an outcomes information storage and 
retrieval system, use of the coding scheme 
described in Appendix A will be necessary. This is 
true especially if one is developing a microfiche 
or computerized system that includes hundreds 
or even thousands of outcome categories. As is 
apparent in Appendix A, the storage system can 
be a one-dimension system, a two-dimension 
system, or a three-dimension system. It can be a 
very simple or a very complex (although fairly 
straightforward in concept) system, depending on 
the need of the local user. 

NCHEMS currently is developing a computerized 
information bank containing selected information 
about outcomes studies and other literary works 
that can contribute to a synthesis or analysis 
relating to the development of outcome indicators 
and measures, correlates of outcome occurrence, 
research questions relating to outcomes, 
strategies for increasing or improving outcomes, 



impacts that can be expected to result from par- 
ticular outputs and combinations of outputs, and 
so forth. This computerized information bank, 
along with a manual card file of literature abstracts, 
illustrates the application of the Structure to 
the process of developing a storage and retrieval 
system for outcomes information. Based on this 
experience, a number of general steps for this 
basic process are outlined below. 

STEP 1 Obtain input from the potential users of 
the system concerning: (a) needs for the 
system, (b) the possible ways they would 
use the system, and (c) criteria the 
system would have to meet to be fully 
applicable to their needs. In addition, 
logically consider what other demands 
on the system might be expected to 
develop once it becomes accepted and 
proves itself. 

STEP 2 Based on Step 1, outline in list form the 
purposes of the system. For example, 
some of the major purposes for the 
NCHEMS system that is being devel- 
oped are: (a) development of a synthe- 
sis of the literature for each measure 
in the NCHEMS Outcomes Measures 
and Procedures Manual (OMPM) that 
relates to its validity and reliability, 
(b) expansion of the OMPM to include 
additional measures in the areas already 
covered and measures in areas not ad- 
dressed previously, (c) expansion of the 
research and scholarship section of the 
Outcomes Structure, (d) development of 
lists of outcome measures and in- 
dicators separately by audience for 
selected type-of-outcome categories in 
the Outcomes Structure, (e) backup 
analysis and support for the develop- 
ment of new student outcome indicators 
and measures, and (f) analyses leading 
to improved application of outcomes in- 
formation to institutional and program 
planning and management. 

STEP 3 Based on Step 2, plus convenience, 
cost, political a;id other considera- 
tions, determine what basic type of 
system it should be, for example, 
manual card files, notebooks of 
organized information sheets, filing 
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cabinets, a manual or automized 
microfiche or microfilm system, an on- 
line computer system, a computerized 
tape system, or a computerized disk 
system. Combination types should also 
be considered, like the NCHEMS ex- 
ample referred to above. 

STEP 4 For each purpose of Step 2, develop a 
list of criteria that the system should 
meet, taking into consideration the input 
from potential users gathered in Step 1. 

STEP 5 Determine whether the Outcomes Struc- 
ture will be the most desirable basic 
framework around which to structure 
and organize the riling system. If it will 
not, decide on a framework to be 
used— whether it is a modified version 
of the Structure or something com- 
pletely different. 

STEP 6 Determine what specific data should be 
entered into the master file and the sub- 
files of the system. For example, at this 
stage of development of the NCHEMS 
system (and this may change later) the 
following items of information about 
each study or other source are being en- 
tered into the master computer file: a 
source code, a unique identification 
number, author/title information, biblio- 
graphic data, postsecondary education 
entity or entities being focused on, type 
of study, type(s) of analysis, its applica- 



tion focus, its assigned NCHEMS Out- 
comes Inventory code number, Its Out- 
comes Structure code number. Also 
determine what form each set of data 
should take, for example, should it be 
coded or written out. 

STEP 7 Design the source-identification, data- 
collection, and data-preparation pro- 
cedures (such as coding activities, 
quality control activities like inspecting 
and editing, and keypunching or other 
recording activities) for the system. 

STEP 3 Design the procedures for entering the 
data into the system, fitting new data 
with the data already there, and 
modifying or deleting data later should 
that become necessary. 

STEP9 Design appropriate procedures for 
manipulating the data within the system 
(sorting, matching, aggregating, tabu- 
lating, reforming, applying analytical 
techniques, and so forth) that will be 
flexible enough to meet all of the 
planned objectives for use of the 
system. 

STEP 10 Design appropriate procedures for 
retrieving the desired and appropriate 
data and analytic or synthesized infor- 
mation from the system, and applying 
those data to the problem at hand. 
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PART IV 

ORIENTING INSTITUTIONAL AND 
PROGRAM FACULTY AND STAFF TO THE 
OUTCOMES STRUCTURE 
AND TRAINING THEM IN ITS USE 



During the CASC Outcomes Learning Task Force 
project and the CASC/NCHEMS joint project 
discussed previously it was found that: (1) Most 
people on the participating campuses had never 
really considered what they were trying to ac- 
complish for students in very systematic, con- 
crete, and specific outcome terms; rather they 
were accustomed to thinking solely in terms of 
activities and tasks they wanted to accomplish, 
although there were undoubtedly outcome 
reasons "in the back of their minds" for wanting 
these activities. (2) People at those campuses, 
and particularly faculty, found it uncomfortable to 
try to talk In systematic, concrete outcome 
terms about what they were trying to accomplish 
for students; and a number of them exhibited 
Initial resistance to trying to apply the Outcomes 
Structure to their situation, although the anxiety 
often seemed to subside as the interview session 
continued. (3) Interviewees had to really strain to 
go from priority outcome categories to especially 
important, specific, concrete, observable out- 
comes within those categories— but with inter- 
viewer support and assistance they were able to 
do so successfully. (4) The orientation materials 
distributed to the interviewees ahead of time 
tended not to be read because they were too bulky 
and complex looking, and the workshop— 
although helpful— was not sufficient for orienting 
campus personnel to outcomes concepts and the 
Outcomes Structure. Clearly, small group 
discussion, or on a one-to-one basis, is needed in 
addition to brief, attractive orientation materials 
and effectively organized and carried out 
workshops. (5) The fact that the CASC/NCHEMS 
project interviews were more successful in intro- 
ducing a cut-down version of the type-of-outcome 
set of categories (prior to going to the entire set) 
than were the CASC project interviews in going 
directly to the entire set of categories, and that it 



was found desirable at Spring Arbor College to 
use an abbreviated set with no code numbers in 
their CASC/NCHEMS project follow-up activities 
on campus, suggests that the full Structure 
should not be presented during the initial stages 
of one's on-campus orientation program. (6) The 
University of Colorado person, who had worked 
with outcomes concepts over a period of several 
years, was able to easily take the Outcomes 
Structure procedures in this document and apply 
them effectively and efficiently without any orien- 
tation or help of any kind. This suggests that once 
they have been properly oriented to outcomes 
concepts and the Structure, campus personnel 
may be expected to use them effectively and ef- 
ficiently, although many probably will still need 
training assistance in how to correctly apply them 
to decision making. 

Based on experiences in the trial projects 
discussed above, plus other experiences at 
NCHEMS and elsewhere, a set of general 
guidelines have been developed and presented 
below for use by those who wish to introduce out- 
come concepts and the NCHEMS Outcomes 
Structure on campus and to have them become an 
integral part of the campus planning, manage- 
ment, and evaluation process. Modification of the 
suggested procedures to meet the local situation 
and conditions is encouraged, and it is hoped that 
the guidelines can be expanded, improved, and 
made more specific as input received from 
people trying to orient their campuses to outcome 
concepts and the Structure (or modified versions 
of it) in different contexts across the country. 



1. Do Your Homework 

Before you can communicate effectively to 
others about outcome concepts and the Out- 
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comes Structure, you must really understand 
them yourself— including cautions that need 
to be taken in using them— and feel at ease in 
discussing them with others. To do this will 
require thorough study of this manual and 
thinking that goes beyond the manual to the 
situation on your campus. It is suggested that 
you also read the conceptual document 
which is a companion piece to this manual, 
designed for those who desire a detailed and 
in-depth discussion of educational outcomes 
and the Outcomes Structure (Lenning, Lee, 
Micek, and Service, 1977). Chapter 2 of that 
document describes six attributes of an 
educational outcome plus five other factors 
that are important for really understanding 
and interpreting a particular outcome; Chapter 
3 contains detailed background discussion 
about each formal dimension of the Outcomes 
Structure and goes into detail about the 
StruciM.e as a whole; and Chapter 4 lists the 
nine basic principles followed in developing 
the Structure (which were also used to 
evaluate it) and discusses the available infor- 
mation that suggests how well these criteria 
were met. If you would also like to compare the 
Structure to other attempts that have been 
made through the years to structure outcomes 
and outcome-related concepts such as goals, 
see Lenning (1977a), which is also available 
fromNCHEMS. 

2. Prepare Draft Copy for a Brief Brochure That 
Effectively Introduces Relevant Outcomes 
Concepts and the Structure. 

To raise initial awareness on campus about 
useful outcomes concepts and the Structure, 
and to bring about curiosity and interest in 
exploring this further, a short, concise, and at- 
tractive introductory brochure will be needed. 
An example brochure or flier from which you 
can build is shown in Appendix D; you may 
prefer an entirely different concept and ap- 
proach for your brochure, however. 

3. Prepare Yourself to Respond to Criticisms That 
Might Be Raised 

You can expect that various people on your 
campus will think of important critical 
questions to raise about outcome concepts 
and the Outcomes Structure, and it is impor- 
tant that you be prepared to respond openly 
and honestly to such questions. Figure 6 lists 
cautions and potential problems that were 



mentioned by interviewees in the joint 
CASC/NCHEMS project. Possible responses 
to the criticisms implied or raised directly here, 
plus criticisms raised by other reviewers of the 
Structure— that are reprinted from Lenning, 
Lee, Micek, and Service (1977)— are presented 
in Appendix E. 

4. Get the Concurrence of Top-Level Manage- 
ment about the Need to Proceed Further 
Top-level administrative p rscnnel— for 
example, the president— may have asked you 
to examine, become familiar with, and explore 
the usefulness of the Structure and outcomes 
concepts like those discussed here, in which 
case you should probably now share your 
thinking at this point with them and make 
recommendations to proceed further or not 
proceed further. If, however, your interest in 
the subject and your exploration in Steps 1-3 
were at your own initiative and you feel 
positive at this point about the usefulness of 
the outcomes concepts and the Structure for 
your campus, now would probably be an ap- 
propriate time to introduce the subject to ap- 
propriate top-level administrators (in a way 
that will interest them in it) and obtain their 
permission to develop some plans for in- 
troducing outcomes concepts and the Struc- 
ture to personnel on your campus. Do not go 
into too much detail in your presentation, keep 
it short, and point out the practical usefulness 
that has been found by people on other cam- 
puses (for example, in the CASC/NCHEMS 
project). Take into consideration the local 
situation and political realities— for example, 
if management by objectives is "in vogue" on 
your campus, show how the Structure can 
contribute to that process; but if it is not, em- 
phasize rather how the Structure can help 
campus people to clarify and stimulate their 
thinking about what they are really trying to 
accomplish in their programs and to delineate 
in concrete terms why they are making use of 
particular approaches and activities. At this 
stage it is important to have obtained 
authorization from the president, or his or her 
designate, to proceed further. 

5. Develop Preliminary Plans and Materials for a 
Campus Orientation Workshop 

For effective dissemination of outcome con- 
cepts and the Structure, a well-designed orien- 
tation workshop will be essential. A possible 



ERLC 



32 



40 



Figure 6 



CAUTIONS AND POTENTIAL PROBLEMS WITH THE NCHEMS 
OUTCOMES STRUCTURE THAT WERE MENTIONED BY VARIOUS INTERVIEWEES 

IN THE CASC/NCHEMS PROJECT 8 



• Need to get students and alumni involved In outcc.T.* planning also. 

• Noed to show people how it will help. 

• This kind of self-analys * Is very important, but will it make any difference on the campus'? 

• Might work if you have the right setting, like a workshop away from the campus. 

• Don't use It unless you have the tlmeand energy to change. 

• One needs time away from day-to-day pressures in order to Implement this. 

• Resistance to the "bureaucratese" in which the document Is written— I don't want to go to the trouble of using 
it unless you can show me that It is directly relet it to my department and its students. 

• Potentially very time consuming— safeguards should be bui't In to insure that the time drain is not too great 
(find out the outcomes of the system that can be achieved In a short time). 

• It may promise more than it delivers. 

• Is the system rea//y hierarchical— is hierarchy the appropriate model? 

• Need funds for the department to develop long-term workshops to incorporate use of the Structure. 

• If it is presented properly, students could become interested in the Structure. 

• I like It. Dynamite! It could be such a long-term thing, however. 

• Difficult to reach consensus on such things— needs a good leader. 

• Takes too much time— what is needed Is funds for a week's ' .aff retreat In a retreat setting. 

• More useful at small colleges than elsewhere because we have more of a chance to see and understand the 
activities which lead to outcomes (the why of outcomes) and we get to know our students very well. 

• Need to start using this in goal setting at the institution-wide ieve' if you start at the department level will go 
in all directions so that when people get together at the institution-wide level will be so far apart will never be 
able to reconcile. 

• It is easier to think of outcomes In specific and concrete terms at the department level, so should start 
applying the Structure to reach con sensusat that level before you try to reach some consensus on 
col lege- vide goals. 

• Need to have total faculty involvement at the departmental level If you are going to make good use of the 
Structure. 

• An excellent instrument if used and the system instructions understood and participated in by most of the 
faculty In the small college. The problem may lie in attitudes— a strong disposition and loyalty to the liberal 
studies and classical structure . . . time constraints, and using the Structure as they envision it. < nee it i* 
learned and Implemented, however, the negatives could be reversed and it couid speed the processes 4 
planning and development. 

• A problem with the Structure concerns differences in definitions, and it includes jargonese. 

• As a student majoring In religious education I tend to think in more general terms. Therefore, it is limiting- 
it limits my options. I would prefer a completely open-end^ d approach. 

• It would be useful to the extent that it serves rather than enslaves. Minimal outcomes should be emphasized. 
Should let more happen bevo. J the stated outcomes. 

• I question its real utility, although it is an asset in that it is systematized. 

• I heard from a staff member who attended the workshop yesterday morning that a lot of good ideas were 
Presented, but that they were perhaps overly idealistic. 

• People need to see the connections sooner to their ongoing tasks. Needed is a week at a retreat setting 
working with faculty on institutional goals and objectives, and how to implement them 



a Reprinted from Lining (197/ uj 
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agenda, sample overhead transparencies, a 
work exercise that is meant to stimulate 
thinking and promote concrete discussion, 
and a role-playing simulation— based on the 
workshops presented at the colleges par- 
ticipating in the joint CASC/NCHEMS trial 
project— are provided in Appendix F. Based on 
these or your own thinking, develop some 
preliminary plans and possible materials for 
use in such a workshop on your campus. Also 
develop preliminary plans for evaluating the 
workshop(s) and for using the evaluation 
data— see Step 9. A majority of the partici- 
pants in the CASC/NCHEMS workshops found 
them interesting and stimulating, and 
especially after certain modifications were 
made, so there is a precedent for expecting 
that such a workshop can be helpful. However, 
follow-up discussion and activities with small 
groups a J individuals *,llbe important In the 
CASC/NCHEMS project, it was not until the 
follow-up interviews with individuals that 
many of those attending the workshop really 
started to understand the Structure and its 
use. interview discussion tended to clarify and 
reinforce the workshop content for them, and 
they felt free to ask questions and to raise per- 
sonal concerns. 

6. Try Out Your Ideas and Draft Materials on 
Trusted Friends 

It will be important to discuss your ide^s, and 
the draft materials developed in Steps 2 and 5, 
in depth with a few appropriate trusted friends 
and colleagues, preferably some off the 
Cv .ipus as well as cn. If students aie to be in- 
volved in the workshop, some of them should 
also be involved in this preliminary tryout. 
Based on the reactions and suggestions 
received in these conversations, make desired 
modifications in your preliminary plans and 
materials. 

7. Establish an "Exploration and Planning" 
Committee concerning a Campus Orientation 
Effort 

The Structure and outcome concepts have 
potentially important applications within all 
major departments and offices on campus. 
Therefore, it would be desirable to form an 
"exploration ^nd planning" committee having 
representat.'/os from the major groups on 
campus (including a student representative). If 



possible, try to limit the size of the committee 
to a number of participants that facilitates in- 
volvement, working together, and effective 
desicion making— for example, many people 
feel that a committee with more than 12-15 
members ' comes unwieldy. In your letter in- 
viting them to participate it would be desirable 
to enclose a copy of the draft brochure, tell 
them about your exploration into the topic and 
your conclusion that the Structure has much 
potential for being of r eal value to your cam- 
pus and its subunits, outline the charge given 
to the committee, and inform them that this 
committee and its efforts have been 
authorized by a directive of the president (An 
initial letter of appointment to the committee 
should be received by them from the president 
a few days before your informational letter, 
which will emphasize to them the importance 
of the committee's work and help motivate 
them to want to get actively involved.) 

8. Reevaluate Your Ideas, Using the Committee, 
and Then Complete Plans and Materials 
Present the ideas, preliminary plans, and 
materials developed to the committee in the 
manner you would use in your orientation 
program, gain their support, and have them 
help you modify and complete the plans as 
needed for an efficient and effective program 
of communication and orientation to the cam- 
pus community. Because the various impor- 
tant sectors of the campus are represented on 
the committee, the members can provide 
valuable views on what is needed to make the 
program effective for the sector they 
represent. Furthermore, they can com- 
municate their interest in outcomes to their 
colleagues and help to carry out the plans for 
their sector of the campus, including the all- 
important follow-up interviews, consultation, 
and other assistance. 

Whether the workshop should be a campus- 
wide event or consist of a series of workshops 
separately for a single or related sectors of the 
campus depends on the local situation (fcr 
example, a very small institution might prefer 
one central workshop). Another possibility is 
to have a central workshop involving selected 
people from each sector and then have these 
people help plan and put on a similar 
workshop— that contains more pertinent case 
examples— for their sectors of the campus. 
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Because on-campus personnel are very busy, 
having the workshop prior to the start of 
school or between terms, extending it over two 
days (for example, from noon to noon), and 
placing it in a retreat setting are desirable. If it 
is necessary to have it during the school year, 
a carefully selected Saturday may be an op- 
tion, or splitting it up into an hour or so a week 
for several weeks may be necessary. 

It is important to have the workshop(s) en- 
joyable and yet serious, fast-paced and yet not 
so rapid-moving that understanding is hin- 
dered, well organized ?nd disciplined and yet 
informal and friendly. 

9. Distribute Brochures and Conduct Workshops 
as Planned— Then Evaluate 
Plans for evaluating the reception of and reac- 
tions to the brochure and workshop(s), for 
determining what more information and 
clarifications are needed by different people, 
and for deteimining the need to make 
available follow-up implementation and 
consultative assistance and/or training 
workshops to help apply the Structure and 



outcomes concepts to actual planning, 
evaluation, and other pertinent problems, 
should have been completed in Step 8. It 
would be useful also to have a specially 
designed questionnaire (tailored to the local 
situation) to be completed immediately after 
the workshops or within several days, plus 
follow-up interviews with individuals or small 
groups. 

10. Make Available and Publicize the Availability 
of Follow-Up Application Assistance 
Based on the evaluation of Step 9 and 
deliberations of the planning committee, 
develop, make available, and let units 
throughout the campus know about the 
availability of assistance in applying out- 
comes concepts and the Structure to par- 
ticular planning, management, and evaluation 
activities. This could include advisory and 
consulting assistance, in-depth training 
workshops for staff who will bo applying out- 
come concepts and the Structure, and direct 
assistance in applying the Structure and out- 
come concepts to help meet pertinent needs. 
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Appendix A 
The Coding Scheme for the Structure 



The successful use of the Outcomes Structure for 
all three basic processes discussed in Part Ih of 
this dojument rests on the fact that for each 
dimension there are successive levels of in- 
creasingly detailed categories. Assigned to each 
category is a code number that has as many digits 
as there are levels of detail in the dimension, 
where the first digit identifies the first-level 
category, the second digit identifies the second- 
level category within the first level category that 
was specified (a zero indicates that none or all of 
them are in luded), and so forth. As discussed in 
Step 8 of Process 1 (page 20), the user of the 
Structure can add additional levels of detail if 
desired, and this adds one digit to each code 
number for every level of detail added. Multi- 
dimensional categories have code numbers con- 
sisting of sets of digits (each identifying a 
category for one of the dimensions) separated by 
decimal points. 

The code numbers provide an abbreviated and 
more economical way of referring to categories in 
the Structure than using descriptive terminology. 
For example, it is much easier to say or write 
"13.1132.22" than "long-duration change in in 
come and standard of living appearing after 
graduation for the family and relatives of students 
and former students." The coding for each 
dimension, and for multi-dimensional categories, 
is discussed separately in the sections that 
follow. 

Coding for the "Audience" Dimension Categories 

The "audience" dimension of the Structure refers 
to the people, the communities, or the things that 
receive or are affected by (or that are intended to 
receive or be affected by) the outcome of concern. 
The categories and subcategories of the audience 
dimension (along with definitions where needed) 
are shown in Tables 1 and 2 on pages 4 and 5. 
Note that there are two levels of detail, with the 
first digit of each code number identifying the 
first-level categories and the second digit iden- 
tifying the second-level categories. Note also that 



within a level, the subcategories are listed in the 
order of a logical (but presumably neutral, 
valuewise) progression, and numbered con- 
secutively. A digit of zero i iJicates that none of 
those categories at this level apply (that this level 
is being bypassed) or that all of them apply. 

If more detail is needed for any second-level 
category (and this may often be the case), an ad- 
ditional level of detail can be added to the dimen- 
sion, as outlined in the discussion of Step 3 on 
page 16. The third-level categories for each 
second-level category would be represented by a 
third digit added to the code number. The third- 
level categories would be ordered within each 
second-level category according to some logical, 
but neutral, progression or else alphabetically, 
and they would then be numbered consecutively. 
A third digit of zero would represent cases where 
the "audience" is broad or unsegmented at th";s 
leve» ct detail. 

Stili nore levels of detail in the audience dimen- 
sion couM be added if needed. A fourth level of 
detail would necessitate four-digit codes, a fifth 
level of detail would necessitate five-digit codes, 
and so forth. 

Coding tor the "Type-ofOuicome" Dimension 
Categories 

The "type-of-outcome" dimension identifies 
whether or not the outcome involves a change in 
status (maintenance versus change), and the 
basic entity that is maintained or changed (such 
as knowledge and understanding, skills and com- 
petencies, attitudes and values, certification 
status, income, standard of living, social interac- 
tions, and art forms or expression). This dimen- 
sion of the Structure has four levels of detail, with 
categories for the first three levels shown in 
Tables 3 and 4 on pages 6-7, and with definitions 
for the categories given in Appendix B (pages 39 
through 51). Categories and definitions for the 
fourth level of detail ("maintenance" versus 
"change") are on page 20. 
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Because there are four levels of detail, each 
category of the type-of-outcome dimension is 
represented by a four-digit code number. Within 
each of the first three levels of detail, the 
categories are ordered alphabetically (except for 
the "other" categories, which are always piaced 
last), and the digit for each level of detail num- 
bered accordingly. The two categories for the 
fourth level of detail go from status quo (1) to 
alteration of the status quo (2), and once again a 
digit of zero indicates that no differentiation is 
being made. 

If there is a need for more detail, additional levels 
of detail can also be added for the type-of- 
outcome dimension, as discussed in Step 9 on 
page 20. A fifth level of detail would necessitate a 
five-digit code number for the categories of this 
dimension, and an extra digit would have to be 
added for each other level of detail added. 

Coding for "Time" Dimension Categories 

The "time" dimension refers to when the outcome 
occurs and/or how long it lasts. Because 
agreement could not be reached on any 
categories and the categories needed vary 
depending on the audience of concern and on the 
situation, no categories were assigned to this 
dimension of the Structure. After evaluating his or 
her need, the user should decide how many levels 
of detail are needed for this dimension, decide on 
categories for each level, order the categories at 
each level (separately for each category at the 
next higher level) alphabetically or in some 
neutral and logical progression, and number each 
category consecutively so that all categories will 
have assigned code numbers. An example of this 
is Table 5 on page 8. 

Coding for Multi-Dimension Categories 

Each of the 33 "audience" categories and sub- 
categories in the Structure can be divided into the 
224 different categories and subcategories of the 
"type-of-outcome" dimension, which results in 
7,392 possible audience/type-of-outcome com- 
bination categories for which information can be 
sorted. The code number for such a combination 
category is the audience code number and the 
type-of-outcome code number separated by a 
decimal point. For example, the code number of 
"change in intellectual skills of students" would 



be 11.2252, and the code numbei of "maintaining 
the economic productivity of the local com- 
munity" would be 31.1311. If one additional level 
of detail were added to both dimensions because 
of local needs for additional specificity, there 
would be three digits in the det of numbers prior 
to the decimal point and five digits in the set of 
numbers after the decimal point. 

If you add categories for the "time" dimension, 
the code number for each three-dimension 
category consists of three sets of numbers 
separated by decimal points, where the code for 
the "audience" dimension is listed first, the code 
for the "type-of-outcome" dimension is listed 
second, and the code for the "time" dimension is 
listed last. Suppose, for example, that a user of 
the Structure developed the following 
student/former student categories and sub- 
categories for "time," and assigned the indicated 
code numbers to each (note that two levels of 
detail are shown): 



10. Short-Duration Outcomes 


11. Short-duration 


12. Short-duration 


outcomes appear- 


outcomes appear- 


ing at or prior to 


ing after gradu- 


graduation 


ation 



20. Long-Duration Outcomes 


21. Long-duration 


22. Long-duration 


outcomes appear- 


outcomes appear- 


ing at or prior 


ing after gradu- 


to graduation 


ation 



The code number for "long-duration change in 
income and standard of living appearing after 
graduation for the family and relatives of students" 
would be 13.1132.22, assuming that the user of 
the Structure did not add levels of detail to either 
the "audience" or the "type-of-outcome" dimen- 
sion. Similarly, 31.3001.21 would be the code 
number for "long-duration knowledge, tech- 
nology, and art form maintenance outcomes 
appearing at or prior to graduation of students, for 
the local community" (an example would be art 
students at the college helping local community 
residents keep local art objects from 
deteriorating). 
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Appendix B 

Definitions and Outcome Measure or Indicator Examples 
For the Type-of -Outcome Subcategories 

Page 



Economic Outcomes 40 

Human Characteristics Outcomes 41 

Knowledge, Technology, and Art Form Outcomes 48 

Resource and Service Provision Outcomes 50 

Other Maintenance and Change Outcomes 51 



The code number for each category is given in the left margin (if the focus is on only 
maintenance the fourth digit would become "1," or if the focus is on only change the 
fourth digit would become "2"). Category definitions are provided in the next column, and 
several examples of possible indicators or measures of such outcomes are provided in the 
right-hand column for each category. The measures and indicators listed are only 
illustrative examples. The majority of examples given are for individuals, but it should be 
remembered that composites of these can often serve as indicators for communities and 
other populations. Future NCHEMS work will include the development of relatively 
comprehensive lists of indicators for selected categories of the structure. 
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Category 
Code Number 


1000 ECONOMIC OUTCOMES" 


1100 


Economic Access and independence Outcomes— Outcomes that relate to the entrance Into obtalnablllty, flexibility, and levels and 
amounts of monetary or pecuniary situations, conditions, and characteristics 




(Categories) 


(Examples of Outcome Measures or Indicators) 


1110 


Economic Access— The amount of openness c ecse of admittance 
to economic opportunities, advancement 


Percentage of students obtaining their first full-time job In the field 
of their choice within a specified time after graduation 

The number of alternatives for an entry level job open to minority 
group graduates compared to minority group nongraduates 


1120 


Economic Flexibility, Adeptebiiity, end Security— The amounts of 
self-sufficiency, liberty, frugality, thrift, self-government, 
confidence, certainty, safeguards, stability, and adjustment that 
are exhibited In economic matters 


Geographic mobility of college graduates compared to those not 
attending college 

Self-report of college graduates about the economic security for 

thorn and tholr familiar anri t ho mnt rlhntlnn nt rnllaoa tn thl« 
i iiui ii aim i noii ion in lug, anu intJ vajiiimuuuvi \ji tsviioyu w ini3 


1130 


income end Stenderd of Living — Amount of profits, return on 
Investment, necessities and comforts of life, wealth, and other 
signs of economic "well-being" that are obtained or possessed 
included is direct support provided to Individuals and the 
community through local purchases by the educational Institution 
and through ataff salutes and wages. 


Amount of annual and lifetime earnings of those attending college 
compared to those not attend Ing col lege. 

Average student and/Or former student reported scores on scales 
measuring perceptions and evaluations of their current and desired 
socio-economic level 


1200 


Economic Resources end Costs— Outcomes that relate to the amount and type of material, energy, effort, people, Organization, and 
other economic assets that are available or that are expend in economic activities and production 




(Categories) 


(Examples of Outcome Measures or Indicators) 


1210 


Economic Costs end Efficiency— The amounts of sacrifice, effort, 
expenditure, and waste present in economic activities and 
production 


The absenteeism and tardiness on-the-job of college graduates as 
compared to nonstudents 

The number of firms that use the college degree as an Inexpensive 
screening device that allows them to hire qualified employees at 
minimum Initial cost to the firm 


1220 


Economic Resources {including employees)— lb* assets available 
that can aid economic production, distribution, and gain 


Percentage of college graduates employed In management 
positions within a specified time after graduation 

Average number of patents and /or copyrights received per student, 
former student, and/Or faculty member. 


1300 


Economic Production— Outcomes that relate to the creation of goods, services, and economic value 




(Categories) 


(Examples of Outcome Measures or Indicators) 


1310 


Economic Productivity end Production— the value of goods and 
services that are created or produced by and within specific 
enterprises of "audiences 1 or clients of the educational Institution, 
and especially In relation to the resources expended In the 
enterprise 


Percentage of college graduates who can adequately do their 
personal typing and complete their own Income tax forms as a 
result of having attended college. 

Expert judges' ratings of the amount of increased worker 
production and higher worker motivation that results from having 
attended college 


1320 


Economic Services Provided— Amount and t/pe of dlrect- 
asalstance activities provided by the educational Institution or Its 
aubunlta In the economic area. 


Dollar amount of goods and services bought In the local 
community by the Institution, Its staff, and Its students Number 
of hours of consultation In the business area provided to area 
companies and Institutions by the university's college of business. 


1400 


Other Economic Outcomes'— An example would be that a company with a large payroll located In this community rather than another 
similar community because there Is a more prestigious college here 


'Many of the aubcategories for this category were auggested by the economic outputs classification developed by Goodman (1971 ) 
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Category 
Coda Number 



2000 HUMAN CHARACTERISTIC OUTCOMES 



2100 



Aspirations— Levels, patterns, and directions (in persons, groups, organizations, or communities) of Interests, desires, drives, 
ambitions, goals, and intentions 



2110 



(Categories) 

Dasiras, Aims, or Qoais— Places, conditions, things, or other ends 
that Individuals and/or groups crave, toward which they have 
ambition, or that they Intend to reach because of Importance to 
them. 



(Examples of Outcome Measures of Indicators) 

Changes In observed desires from college entrance to graduation 

Changes in the reported aspirations for graduate school as a class 
proceeds through undergraduate school 

Self-repo.1 of changes In goals and aspirations as a result of 
college 



2120 



Disiikas, Likas, and mferesfs— The persons or types of persons, 
objects, content areas, occupations and other things and 
situations for which there Is a preference or antipathy 



2130 



Motivation or Driva lew/— The Intensity of striving toward a goal 
tnat Is elicited by a need or other stimulus 



2140 



Othar Aspirational Outcomas 



The reported likes and dislikes of persons before college as 
compared to after graduation, and comparison with such change 
over the same period of time for those the same age wt att*~ Jlng 
college. 

Score or change In score on an Interest Inventory, e g., Strong 
Vocational Interest Blank, Kuder General Interest Survey, Kuder 
Occupational Interest Survey, ACT Interest Inventory.* 

Self-report of changes In Interests as a result of college 



Score or change In score on an Instrument that measures "need for 
achievement" or "achievement motivation," e g , the Strong 
Vocational Interest Blank Academic Achievement Scale, the 
College Student Questionnaire Motivation for Grades Scale, the 
California Psychological Inventory Achievement Scales, Personal 
Value Inventory 

Self-report of changes In motivation level as a result of college 



2200 



Compefer>ceand Skills— Levels, patterns, and direction of ability, capability, proficiency, and talent of different kinds 



2210 



(Categories) 

Acadamic Skills- The amount of ability or competence In taking 
tests, earning good grades, persisting In college, etc without 
regard to tha amount of cognitive learning that has taken place 



2220 



Citizanship and Family Mambarship Skills— The ability or 
competence to perform relative to the rights, duties, and privileges 
of a member of a family, community, atata or nation, for example, 
competence In managing family finances, being an effective 
consumer, and evaluating political issues 



(Examples of Outcome Measures or Indicators) 

Grades earned when the effect of ability, motivation, and other 
*uch factors have been controlled 

Persistence In college when the effects of ability, motivation, and 
other such factors have been cancelled out 

Score or change In score on a test of study skills, e g , Brown- 
Hoitzman Survey of Study Habits and Attitudes, Comprehensive 
Test of Basic Study Skills 



Self-report of abilities pertaining specifically to citizenship and 
home membership that college accentuated 

Evaluation by others of citizenship and home membership skills 
mastery exhibited 

Score or change In score on the Vlneland Social Maturity Scales 



* Aa an example, one could at graduation compare Interest teat scores of college atudenta to a group of their high school claaamataa not attending 
collage who had similar internets in high school. As another example, one could look at Interest test change scores for college atudenta, adjuated 
for Initial level. 
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Category 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2230 


Cmstivity Skins— The amount of ability or competence In 
designing, producing, or otherwise bringing Into existence Original 
perspectives, explanations, and Implementations 


Score or change in score on a test that measures originality and 
creative ability, e g , Minnesota Test of Creative Thinking, Test of 
Creative Ab'Hty, GuH'ord's Alternate uses Test, Sixteen 
Personality Factors Questionnaire Creativity Scale 

Evaluation by judges of creative ability demonstrated In a building 
or forming task 


2240 


XTwnpM«#i/iM P/\miHniii#H*ffAii OAr iiis TKa anr\s% lint f\t Mhllltxi nr 
CXPfwaSIOn mnO xJOmfnUfiiCmllOn o«/H3 — 1 rw WTrOUni Oi •Ollliy or 

competence In conveying information, attitudes, or emotions on a 
one-to-one basis and/or to large or small groups or populations, 
by whatever media, In order to Inform, challenge, uplift, and/or 
persuade, etc., and In receiving and Interpreting such 
communications— through reading, writing, speaking, listening, 
touching, body movement, silence, and cultural arts like acting, 
painting, sculpturing, singing, playing musical instruments, etc 


Score or change in score on tests that measure the ability to 
communicate or express oneself 

Judges' rating in a debate or speech contest 

Judges' rating of expression in a music, art, or ballet contest 


2250 


infosctuti Skills— The amount of ability or competence in 
formulating and analyzing problems, comprehending and 
understanding, synthesizing information evaluating information. 
Implementing a solution to a problem, and In locating, retaining, 
and filtering relevant knowledge 


Score or change in score on a test that measures ability to analyze 
and solve problems and to make inferences, e.g., California Test of 
Mental Maturity, Watson-Glazer Critical Thinking Appraisal, 
California Psychological Inventory Intellectual Efficiency Scale 

Self-report of changes In analytical ability as a result of college 


2260 


htfpfsonti, Lttdtship, and Organ/zaf/ona/ Skms— The amount 
of ability or competence In effectively living and Interacting with 
others, social organizing, being a congenial friend and companion, 
establishing courses of action for others, and influencing others to 
follow 


Leadership awards 

Self perceptions and evaluation of interpersonal and leadership 

sunny 

Perceptions by judges of interpersonal and leadership skills 

Score or change In score on a test that measures leadership and 
interpersonal ability, e g , California Psychological Inventory 
Leadership Scale. Chapin Social Insight Scale. 


2270 


Occupaf/ona/ Skins— The amount of ability or competence In the 
special, unique skills required by particular occupations, and in 
seeking, gaining, and maintaining a particular level and kind of 
employment 


Spatial relations test scores for someone whe is, or Is going to be. 
an artist 

Demonstrated ability in writing FORTRAN or COBOL for someone 
who is, or Is going to be, a computer programmer. 

Score or change in score on the Bennett Mechanical 

Comprehension Test. 


2280 


Physical and Motor Skills— The ability or competence In tasks 
requiring physical coordination, dexterity, manipulation, and other 
muscular or motor skills 


Score or change in score on tests that measure motor skills, e.g . 
Crtssey Dexterity Test, Minnesota Rate of Manipulation Test 

Judges' scores on skill events in athletic competition such as 
gymnastics, diving, and figure skating 


2290 


Ofner Skill Outcomes— Examples are the ability to teach 
effectively, to handle one's leisure, etc. 




2300 


Morsm, Satisfaction, snd A ftsctivChtractfistics— Levels, patterns, and directions of character' Mies typified by emotion 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2310 


Attitud** — The disposition or tendency to respond either 
positively or negatively to particular persons or types of persons, 
things, situations, etc. It Is a predisposition to act In a certain way 
AND 

Va/uea— A strong preference based on a conception of what Is 
desirable, important, and worthy of esteem. Values affect an 
Individual's actions and thoughts toward others. 


Score or change in score on an attitude scale, e g . Thurstone and 
C have's Scale for Measuring Attitudes Toward the Church, College 
Student Questionnaire Part 1. Adorno Ethnocentriam Scale, Shaw 
and Wright Scales for the Measurement of Attitudes 

Self-report of one's attitudes and the effect of college on them. 

Score or change In score on an Instrument that assesses values, 
e g , Atport-Vemon-Lindsey Study of Values, Differential Value 
Profile, Work Values Inventory. 

Self-report of one's values and the effect of college on helping to 
clarify them 




Category 
Cods Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2320 


Beiiets, Commitments, end Philosophy of Lite— The acceptance 
and Internalization of particular propositions or declarations, the 
particular things that one Is convinced are true The held view of 
what "man" Is, the purposes and reasons for a person's existence, 
and the system of principles and laws that should govern his/her 
thought, morals, character, and conduct or behavior Included Is 
the promotion of and the adherence to the conventions, practices, 
and teachings of religious organizations or sects 


Score or change In score on Instruments that assess beliefs, e g , 
Harvey's Conceptual Systems Tsst, Inventory o' Beliefs 

Self-report of one's beliefs and commitments and the effect of 
college on them. 

The membership and participation In, and support of, a particular 
religious Organization or cause prior to as compared with after 
college 

Self-report of one's philosophy of life and the effect of college on 
clarifying and organizing It 


2330 


Feelings end fmof/ons— The disposition or tendency to respond or 
not respond subjectively to stimuli and the ability ** control or not 
control such expressions, I e , feelings of anguish or distress, 
anticipation, anxiety, concern, contentment, empathy, excitement, 
fear, frustration, happiness and Joy, humor, lethargy, love, 
pleasure, satisfaction, sorrow, etc 


OfMrinASS And Af^mntAnnit nf f Mil Inn ft hAfrir* rrtilnrui ratmrtimri tn 
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after college. 

Development of an apprecietion of different cultures and a wide 
range of human values as a result of college 

Greater reported satisfaction with life as a result of college 


2340 


Mores, Customs, end Stendards of Conduct— Social and cultural 
practices, rules, and conventions designed to guide personal and 
corporate behavior They have strong ethical or moral significance 
according to tradition and are enforced by social disapproval of 
violations 


Self-report of the effect of college on assimilation or internal); lion 
of the customs of community or society. 

Score or change In score on the California Psychological Inventory 
Socialization Scale. 

The adherence to particular mores or social customs prior to 
col lege as compared to after col lege . 

The amount of subjectivity and emotion guiding one's standards of 
conduct prior to college as compared to after college 


2350 


Other Affective Outcomes 




2400 


Perceptual Chertcteristics— Levels, patterns, and directions of consciousness, awareness, and sensitivity exhibited, and the view(s) 
orconcept(s) of self, others, surroundings, events, Ideas, etc. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2410 


Perceptuei Awareness end Sensitivity— The amount of conscious- 
ness or awareness o', or sensitivity to, stimuli that are exhibited by 
individuals or groups. 


Increased sensitivity to needs and emotional cues provided by 
others 

Increased alertness to the opportunities confronting one. 


2420 


Perception of Self—The view held about oneself, the character- 
istics that are perceived, l e.. self concept 


Development of positive self-regard and self-confidence aa a result 
of college 

Score or change In score on a self-concept scale, a g., Adjective 
Check List, California Psychological Inventory Self Acceptance 
Scale, Tennessee Self Concept Scale. 


2430 


Perception of Others— The manner In which other Individuals and 
particular groups of others are viewed or perceived, the 
characteristics that are perceived. 


Reports by observers about how a person's respect for others haa 
changed as a result of college 

Sei '-report of how one's view of others has changed as a result of 
college 


2440 


Perception of Things— The view one holds (I s., the characteristics 
noted) of Ideas or other things being examined with the physical 
senses 


Increased respect for the ideaa of others aa result of college. 

Movement as a result of college experiences from seeing things as 
all "black and white" to complex "grays " 


2450 


Other Perceptuei Outcomes 


1 
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Otegory 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2500 


Personaht) and Personal Coping Characteristics— Levels, patterns, and directions of human conditions, factors, and traits related 
specifically to the mind and mental processes (other than skills. Knowledge, and understanding). 




/ f a * mnf\r 1 mm \ 


(Examples of Outcome Measures or Indicators) 


2510 


Advent urousness and Initiative— Willingness to take chances and 
risks, how daring an individual is, willingness to take a stand or 
speak out, willingness and capacity to Initiate personal action or to 
become actively involved 


Reports by impartial observers of changes in initiative that seem to 
have resulted from college attendance. 

Self-report of the effect of college on one's willingness to take a 
chance, e g , to take an educated guess on an exam 

The frequency that one exhibits speaking out on issues as the 
college career progresses 


2520 


Autonomy and Independence— The amount of freedom from 
control and influence of others that is exhibited 


Score or change in score on personality scales that measure 
autonomy and independence, eg., Sixteen Personality Factors 
Questionnaire Group-Dependent vs Self-Sufficient Scale, 
Edwards Personality Inventory Independent In His Opinions Scale, 
College Student Questionnaire Independence Scales, Omnibus 
Personality Inventory Autonomy Scale 

Self-report of willingness to volunteer or "stand up for one's rights" 
and the effect of college attendance on such willingness. 


2530 


Dependability end Responsibility— The amount of reliability, 
trustworthiness, and accountability for own behavior that Is 
exhibited 


Reports by observers of changes In dependability and 
responsibility that have occurred during college. 

Score or change In score on scales that measure dependability and 
responsibility, e g., California Psychological Inventory Responsi- 
bility Scale, Edwards Personality Inventory Assumes Responsi- 
bility Scale, Sixteen Personality Factors Questionnaire Expedient 
vs Conscientious Scale 


2540 


Dogmatfsm, Authoritarianism, and Open-Mindedness— The 
amount of o pen-minded neas, assert Iveness, unassertlveness, 
and/or unquestioning obedience to authority that Is exhibited 


Reports of expert observers about changes In open-mi ndedness 
that have taken place during college. 

Score or change In score on a scale that measures dogmatism 
and/or authoritarianism, e.g., Rokeach Dogmatism Scale, 
California Psychological Inventory Dominance Scale, Omnibus 
Personality Inventory Religious Orientation Scale 


2550 


Flexibility and Adaptability— The amount of adjustment to new and 
changing situations and circumstances that Is exhibited. 


Score or change In score on a scale that measures flexibility, e.g., 
California Psychological Inventory Flexibility Scale, Omnibus 
Personality Inventory Practical Outlook Scale, Sixteen Personality 
Factors Questionnaire Practical vs. Imaginative Scale. 

Reports by observers of changes in adaptability and flexibility that 
have occurred during college. 

Self-report of the effect of college on adaptability and flexibility. 


2560 


Habits— The tendency to perform certain actions or to behave In 
characteristic, automatic ways 


Observations by others of changes In habit orientation that have 
occurred during attendance 

Self-report of changes in habits that have resulted from college. 


2570 


Psychological Functioning—The amount of psychological 
adjustment, contact with reality, self-understanding, and self- 
actualization (optimum self-realization) that is exhibited 


The amount of realization of one's actual strengths and 
weaknesses, and of what is real Ity . 

Score or changes In score on an Instrument that measures 
psychological adjustment, eg., Minnesota Multiphasic Personality 
Inventory, Sixteen Personality Factors Questionnaire, Moody 
Problem Check List 

Reports by expert observers about changes In the psychological 
functioning of individuals that have occurred during college 
attendance 
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Category 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (Continued) 


2680 


Tolerance and Persis fence — The amount of endurance, tenacity, 
forbearance, patience, and restraint that la exhibited 


Observations by others of changes in tolerance and persistence 
during college 

Score or changes In score on an instrument that measures 
tolerance and persistence, e g , Edwards Personality Inventory 
Persistence Scale, California Psychological Inventory Tolerance 
Scale 


2590 


Other Personality and Personal Coping Outcomes 




2600 


Physical and Physiological Characteristics— Le*e\9, patterns, and directions of human body traits and processes (other than skill 
functioning) 




(Categories) 


(Examples of Outcome Measures or Indicators) 


2610 


Physical Fitness end Traits— Physical and physiological 
characteristics such as toughness, endurance, strength, speed, 
flexibility or dexterity, physical energy, muf^lar control, size, 
vocal characteristics, etc 


Score or change in score on physical fitness tests, e.g., AAHPER 
Youth Fitness Tests, Basic Fitness Tests 

Self-report of "feeling in better physical shape" as a result of 
college. 


2620 


Physiological Health— The physical well-being of individuals, how 
well the system of normal bodily operations Is functioning 


r.'edical doctor's health physical examination report at college 
entrance compared to at college graduation 

Self-report of the effect of college attendance on how well alumni 
take care of their bodies 


2630 


Other Physical or Physiological Outcomes 




2700 


Status, Recognition, and Certification— levels, patterns, and direction concerning recognition of accomplishments, power, prestige, 
reputation, etc 




(Categories) 


(Examples of Outcome Measures or indicators) 


2710 


Completion or Achievement Award— A certificate, diploma, or 
some other award for having completed a course or program, for 
some demonstrated proficiency, or for accomplishment of some 
type 


An honorary degree 
Graduation diploma. 
Alumni achievement award 
Sales award or a job promotion 
Oanforth Fellowship Award 
Being named a Rhodes Scholar 


2720 


Credit Recogmt ion— Formal or Informal acknowledgement of work 
completed or of confidence, trust, approval, etc 


Graduate school grades. 

Credit hours given for completing a course 

By-line credit for a movie, play, book, or article 

Financial credit rating Issued by a bank or credit bureau 


2730 


Image, Reputation, or Statue- The amount of fame, distinction, 
respect, and standing In the eyes or the profession, the 
community, or some other group. 


Being on the social register. 

Being listed In Who's Who. 

Oral and written acknowledgements from others 

Being interviewed by tha press, radio, or TV 

Writing an autobiography that Is published or having a biography 
written about you. 


2740 


Licensing and Certification— Formal written authority that a 
person or firm Is qual'fled and has met the test to practice some 
skill or speciality occupation 


Entry Into the state bar 

Passing a cosmetology licensing exam 

Being a certified public accountant 

An insurance company that has been licensed to sell In a state. 
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Category 
Coda Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2750 


Obtaining e Job or Admission to a FoHowUp Program— Success 
In being selected for a postgraduate employment position or e 
special educational program at a higher level 


Entrance to a university after graduation from a community 
college 

Entrance to law, medical, or graduate school 

Being selected by the civil service 

Being selected for a company executive position 

Being hired In the special field fot which the training applied 


2700 


Power and/or Authority— The amount of acknowledged 
authorization or ability to influence, command, enforce obedience, 
or set policy as a right of rank, position, delegated jurisdiction, 
skill, strength, wealth, ate 


Appointment or election to e position of authority. 
Earn ing promotion to a position of authority 
Influencing important community or public decisions 


2770 




Getting acknowledged credit for the important job having gotten 
done 




Job, School, or Lift Success — Evidence of success In one's 
occupation or career, In graduate or professional school, or In 
some other aspect of one's Ufa that Is covered In any of the above 
categortee. 


Self report of success in career 

Teacher's rating of success In graduate school 

Employer's rating of Overall on-the-job performance 


2780 


Other Stetus, Recognition, end Certification Outcomes 




2800 


Sociei Activities end Roiee— Levels, patterns, and directions of social functions assumed and carried out 




(Categories) 


(Exemples of Outcome Measures or Indicators) 


2810 


Adjustment to Retirement— Altering self and lifestyle to meet the 
needs and adapt to tha limitations of tha retirement years 


Percentage of college educated retirees reporting productive 
retirement years compared to reports of those who never attended 
college. 

Self -report of the effect of hevlng ettended college on the 
retirement years. 


2820 


Affiliations— Finding appropriate organizations and institutions to 
join and associate with . and being accepted by them 


Number of affiliations and changes In ef fillet ions for college 
graduates as compared to those never attending college. 

Self-report of the effect of having attended college on tha 
af f 1 1 let ions sough t and on the af f 1 1 1 at Ions won 


2830 


Avocetionei end Sociei Activities end Roiee —Finding, pursuing, 
and achieving rewarding nonwork activities, hobbles, and parts to 
play In society, and exhibiting that pattern of behavior that Is 
expected of persons having tha stetus that has been earned . 


The sociei roles end evocations of college graduates as compared 
to those who never ettended college 

Self-report of the effect of having attended college on the 
avocational and social roles sought, end on those practiced. 


2840 


Cereer end Voce ti one i Activities end Roles— Exhibiting the 
patterns of behavior expected and/or that are needed for the pert In 
the ''world of work" that ha* oeen accepted or entered Into. 


The career roles of college graduates as compared to those who 
never ettended co I lege. 

Reports of employers concerning the advancement and roles of 
college trelned employees versus the advancement end 
occupational rotes of those who never ettended college. 


2860 


Citizenship Activities end Rotes— Facilitating and contributing to 
govarnmentel functions end to the overall well-being of 
Individuals, the community, and larger society. 


Percent voti ng in e municipal or state election 

Flnencisl end other contributions given to service organizations 

Percent running for public office cr campe gnlng for someone who 
is 
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Category 
Code Number 


2000 HUMAN CHARACTERISTIC OUTCOMES (continued) 


2860 


Family Activiti9$ and Roles— Contributing to end facilitating 
family functions, le, parent roles, sibling roles, son/daughter 
roles, etc 


The family roles of college graduates as compared to those who 
never attended col lege 

Self-report of effect of the college on the roles played in one's 
family 


2870 


Friendships and Reiat ionships— Socially Interacting with and 
entering Into and sustaining intimate, In-depth, and satisfy 
associations with others 


Characteristics of friends and relationships of college edirited 
people versus those never attending college 

Self-report of the effect of college on friendships and social 
relationships 


2880 


Other Activity and Rote Outcomes 




2900 


Other Human Characteristic Outcomes 
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Category 
Code Number 


3000 KNOWLEDGE, TECHNOLOGY, AND AH T FORM OUTCOMES 


3100 


Genera/ Knowledge end Unders t ending *— Fern II larity with, analyse and con —ehenslnn Of, and application o< fccts and principles 
across broad ereas of study— breadth of knowledge end understanding—as a result of dissemination through educational teaching- 
learning activities 




(Categories) 


(Examples of Outcome Measures or Indicators) 


3110 


Knowledge end Understending of Qenerel Facts ard 
Terminology — Knowlnc ebout end understending, end hevlng en 
adequete vocabulary to be able to describe, the reality, existence, 
end circumstances of particular sensory (observed, heard, felt, 
etc ) phenomena, objects, people, products, events, conditions, 
etc , or components thereof. 


Students' scores or changes in score on standardized or classroom 
tests that measure knowledge and understanding of general 
terminology and /or facts. For example, the Miller Analogies Test 
focuses entirely on knowledge ano understanding of general 
terminology, and *ests like the College .evei Examination Program 
(CLEP) or the Graduate Record Exam (GRE) general exam Include 
coverage of general terminology and facts 

Students' self-report of Knowledge and understanding about 
general terminology and facts 


3120 


Knowledge end Understending of General Processes— Knowing 
ebout and understending customs, rules end standards for 
judgments, guidelines, processes, methods, procedures, 
techniques, trends, end other ways of applying and making use of 
terminology end facts. 


Students' scores or changes In score on standardized or classroom 
tests measuring comprehension of general conventions, 
processes, and methodologies 

Students' grades In a general application survey course 


3130 


Knowledge end Understending of Qenerel Theory— Knowing about 
and understanding principles and generalizations, theoretical 
formulations, hy[ Dtheses, supposition, conjecture, etc 


Students' scores or changes in score on standardized or classroom 
tests measuring comprehension of general theories In a oad field 
of study 

Students' grades in a general survey course on theories of 
philosophy 


3140 


Other Qenerel Knowledg \nd Understending 




3200 


Specialized Kno Pledge end Understending* —Familiarity with, analysis and comprehension of, and application of facts and principles 
In particular specialized fields of study— depth of knowledge end understanding— as a result of dissemination through educational 
teaching/learning activities. 




(Categories) 


(Examples of Outcome Measures or Indicators) 


3210 


Knowledge end Understanding of Specialized Facts and 
Terminology— Knowing about end understending, end having en 
adequete vocabulary to be able to describe the reality, existence, 
end circumstances of particular sensory (observed, heard felt, 
etc ) phenomena, objects, people, products, e»cr?« renditions, 
etc , or components thereof 


Students' scores or changes In score on standardized or classroom 
tests that measure knowledge and understending In e narrov, 
specialized area of study Profeeslonal certification and licensing 
exams usue'ly focus on this type of knowledge, as do tests like the 
College Level Examination Program (CLEP) subject exams or the 
Graduate Record Exam (GRE) area exams. 

Students' self-report of knowledge end understanding about 
specialized terminology and facts 


3220 


Knowledge and Understanding of Specialized Processes— Knowing 
about end understanding cuatoms, rules end standards for 
judgments, guidelines, processes, methods, procedures, 
techniques, trends, end othec weys of applying end making use of 
terminology and facts. 


Students' scores or changes in score on standardized or classroom 
tests measuring comprehension of conventions, pfAoessss, 
methodologies, and techniques unique to particular > < "iaMzed 
professions or disciplines 

Students' grades In a specialized professional course or program . 


3230 


Knowledge and Understanding of Specialized Theory -Knowing 
about end understending principles end genetdllzations, 
theoretical formulations, hypotheses, supposition, conjecture, 
etc 


Students' scores or changes In score on standardized or classrc 
fists measuring comprehension of specialized theoretk 
foi nutations ar 1 models 




Students' grades in a course that goes Into depth about one or 
more theories or models unique to a specialized dhclpl'ne or 
profession 


3240 


Other Specialized Knowledge and Understanding 




3300 


Research and Scholarship— Knowledge end understanding, techniques, end physical products resulting from basic and applied 
research end scholarship. 



'The subcategories used for this category came from Bloom (1956) 



I Category 
Code Number 


3000 KNOWLEDGE, TECHNOLOGY, ANO ART FORM OUTCOMES (continued) 




(Categories) 


(Examples of Outcome Measures or Indicators) 


3310 


Research end Scholarship Knowledge and Understanding— The 
discovery, development, preservation, and professional dissemin- 
ation of knowledge and understanding resulting from activities 
conducted in basic and applied research and scholarship 


Average number of basic research publications, applied research 
publications, textbooks, or monographs etc , per student, former 
student, and/or faculty member over a specific period of time 

Number of faculty members and /or former students In the 
sciences listed In American Men of Science 


3320 


Research end Scholarship Products— Applied techniques (for 
example, a new therapy treatment in the field of medicine or a new 
technique In the field of music) and physical products (for 
example, a new or refined serum) developed from basic and/or 
applied research and scholarsh ip. 


Average number of patents and /or copyrights received per student, 
former student, and/or faculty member overa given period of time 

Average number of awards and citations received per student, 
former student, and /or facutly member (over a given period of 
time) for discovery or development of technological products. 


3400 


Art Forms end Works— Reproducing and preserving existing artistic ferns and works, and developing new or revised anlstlc forms and works 




(Categories) 


(Examples of Outcome Measures or Indicators) 


3410 


^hitecture— Outcomes Involving the design for construction of 
buildings, landscape, living complexes, etc. 


Number of architectural works completed by students, former 
aiuueno, end /or faculty 

Number of awards and other recognitions received for architectural 
works on the campus commissioned by campus officials. 


3420 


Dance-Outcomes Involving preservation or development of 
forms, works, and performances In Lie art of dance. 


Number of former students receiving recognitor for performances 
In this area 

Number of students Involved In dance auditions and public 
performances. 


3-w 


Debate end Oretory— Outcomes Involving preservation or 
development of to' is and performances En the oratory arts. 


Competition record over a period of years of the college's debate 
team 

The average number of graduates each year who go on to some 
Kind of oratorical career 


3440 


Drama— Outcomes Involving tr j preservation or development of 
forms, works, and performances In the professional and amateur 
theatrical arts. 


The number of students who enter a professional acting career, 
and the number acting on an amateur basis. 

The number of drama performances put on for the local community 
each year. 


3450 


Literature and Writing — Outcomes Involving the preservation or 
development of forma and works In the production c? prose, verse, 
and other writings. 


The average number of literary works each year published by 
students, former student*, and /or faculty members 

The number of students and faculty —eh vtv *hn k*va «ntwwi • 
formal s* ate or national writing competition 


3460 


Music— Outcomes Involving the preservation or development of 
forms, works, and performances In the professional and amateur 
theatrical arts 


The number of musical productions put on each year by the college 
that are open to the public. 

The number of stud* volved In public music recitals and other 
performances 


3470 


Peinting, Drewmg, end Photography— Outcomes Involving the 
prest vat ion or development of forma or works in the graphic and 
pictorial arts 


The number of paintings, and their quJIty In the campus art 
gallery. 

The number of awards won over a certain period of time for 
pictorial works by students, former students, and faculty 
members , 


3480 


Sculp turn— Outcomes Involving the preservation or development of 
forms or works In the carving, chiseling, casting, modeling, or 
other sculpturing areas 


The number of sculptures that have been commissioned by the 
college and placed throughout the campus 

The forms oi sculpture that have been developed on the campus 


3490 


Other Fine Arts 




350C 

L 9 1 


Other Knowledge, Technology, end Art Form Outcome* 
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Category 
Code Number 


4000 RESOUPCE AND SERVICE PROVISION OUTCOMES 


4100 


Provision of Facilities and Events— The availability, use, and participation In campus happenings, buildings, equipment, and other 
resources by students, other Individuals, and particular groups or communities 




(Categories) 


(Examples of Outcome Measures or indicators) 


4110 


Provision of Facilities— Availability and use of campus grounds, 
buildings, rooms, equipment, etc 


Number of facilities made available to the students during a 
particular period of time 

Total number of hours each facility was used by people in the 
community, and the number of people-hours of use over a specific 
period of time 


4120 


Provision or Sponsorship of Events— Availability arid participation 
in happenings on the campus or off that sre provided or stlmulsted 
by the col lege or one of its components 


The numoer of people who attended athletic events, cultural 
events, or other events 'xovided and/or sponsored Ly the college 
In any one year 

The number of column inches of newspaper coverage received by 
specific events in local, regional, and national newspapers 


4200 


Provision of Direct Services— The availability, use, and receipt by students, other Individuals, and particular groups or communities of 
assistance, care, or other service 




(Categories) 


(Examples of Outcome Measures *v Indicators) 


4210 


Teaching*— Activities and programs designed to Instruct and to 
Imparl knowledge, skills, attitudes, etc. 


Average number of courses taught and number of contact hours 

per semester In the regular program. 

Extension courses provided in any one calendar year. 


J99D 


Advisory and Analytic A ss/sfsnce— Activities and programs 
designed for the purpose of (upon request) offering suggestions, 
recommendations, counsel, Information, calculations, and 
studies 


Number of advisory and analytic assistance services offered to 
students, staff, and/or to the public 

Number of person-hours spent by staff in providing this assistance 
over a specific period of time. 


4230 


Treatment, Care, and Referral Serv/ces— Helping and direct 
assistance services, other than those above, provided by the 
Institution, institutional units, and/or Institutional staff 


The treatment, care, and referral eervlcee offered by the Institution 
and its staff, and health services, day care for children of working 
mothers, counseling, crisis referral, and drug treatment and the 
amount these services are used. 

The reported satisfaction of users of these servloee with the 
treatment and care received. 


4240 


Provision of Other Services— An example would be direct civic 
leadership provided to the community. Another example would be 
offering keypunching service* 


4300 


Other Resource and Service Provision Outcomes— An example would be trie attention and good will the college draws to the local 
community because It Is seated there. 




•Some people would consider teaching to always be a producer/facilitator activity that leads to outcomes Others would, however, consider 
teaching to be an ojtput that results from the interaction of faculty, equipment, students, and other educational reaourcee Those who hold the 
first viewpoint should fust Ignore this category 
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Category 
Code Number 


5000 OTHER MAINTENANCE ANP CHANGE OUTCOMES 


5100 


Aesthetic-Cultural Conditions— Preserving or bringing about changes in tastes, level and Kinds of aesthetic-cultural emphasis, 
aesthetic-cultural availability and opportunities, aesthetic-cultural activity and participation, etc 


5200 


Organizational Format, Activity, end Operations — For organizations, groups, and systems (and their components), maintenance or 
change In organizational communications, operational methods and Interaction, operational effectiveness, organizational 
relationships, organizational arrangement and configuration, organizational activities and programs, and other such organizational 
characteristic outcomes 


5300 


Other Maintenance and Change— Outcomes not covered by any of the other subcategories of "Maintenance" and "Change" In this 
dimension of the Outcomes Structure An exam pie might be "destruction of life support in the environment " 



O 51 

ERIC 58 



Appendix C 

An Example of Outcomes Classified with the Structure Using 
Outcome Statements from the NCHEMS Outcomes Inventory 



OUTCOME CATEGORY OF 
THE NCHEMS INVENTORY 



CORRESPONDING CATEGORY OR CATEGORIES 
OFTHE OUTCOMES STRUCTURE TYPE-OF-OUTCOME DIMENSION* 



1 1 1 00 Knowledge Development 

1 1 1 01 General Knowledge 
1 1 1 02 Specialized Knowledge 



1 1 2 00 Skills Development 

1.1 2 01 Application Of Knowledge Skills 

1 1 2 02 Critical Thinking and Reasoning Skills 

1 1 2 03 Creativity Skills 

1 1 2 04 Communication Skills 

1 1 2 05 Motor Skills 

1 1 3 00 Knowledge and Skills Attitudes, Values, and Beliefs 

1 1 301 Intellectual Disposition 

1 2 1 00 Social Skills Development 

1 2 1 01 Interpersonal Participation 
1 2 1 02 Leadership 
1 2 1 03 Citizenship 

1 2 2 00 Development of Social Attitudes, Values, and Beliefs 

1.2 2 01 Political 

1 2 2 02 Racial/Ethnir 

1 2 2 03 Personal Ethics 
1 2 2 04 Social Conscience 

1 2 2 05 Socioeconomic Aspirations 

1 3 1 00 Student Health Development 

1.3 1 01 Physical Health 
1 3 1 02 Mental Health 

1 3 2 00 Student Personal Attitudes, Values, and Beliefs 

1 3 201 Religious and Spiritual 
1 3 2 02 Change/Stability 

1 3 2 03 Self-Concept 

1 4 1 00 Career Preparation 



1 4 1 01 Academic Preparation 
1 4 1 02 Vocational Preparation 



1 4 2 00 Career Attitudes, Values, and Beliefs 

1 4 201 Achievement Orientation 
1 4 2 02 Educational Aspiration 
1 4 2 03 Educational Satisfaction 
1 4 2 04 Vocational Aspirations 

2 0 0 01 Discovery of New Knowledge 

2 0 0 02 Interpretation and Application of New Knowledge 
' U 0 03 Reojganlzatlon ^ New Knowledge 

(Continued) 



3002 Knowledge, Technology, and Art Form Change Outcomes 
3102 Change In General Knowledge and Understanding 
3202 Change In Specialized Knowledge and Understanding 



2202 Change In Competence and Skills 

2252 Change In Intellectual Skills 

2252 Change in Intellectual Skills 

2232 Change in Creativity Skills 

2242 Change In Expression and Communicate.- skills 

2282 Change in Physical and M-»ur Skills 

2312 Change In Attitudes and Values 

2322 Change In Beliefs, Commitments, and Philosophy of Life 

2122 Change In Desires, Alms, or Goals 

2262 Change In Interpersonal, Leadership, and Organizational Skills 
2262 Change In Interpersonal, Leadership, and Organizational Skills 
2262 Change in Interpersonal, Leadership, and Organizational Skills 
2222 Change In Citizenship and Family Membership Skills 

2312 Change In Attitudes and Values 

2322 Change in Beliefs, Commitments, and Philosophy of Life 

2312 Change in Attitudes and Values 

2312 Change In Attitudes and Values 

2432 Change In Perception of Others 

2312 Change In Attitudes and Values 

2312 Change in Attitudes and Values 

2322 Change in Beliefs, Commitments, and Philosophy of Life 

2102 Change in Aspirations 

2622 Change in Physiological Health 
2572 Change in Psychological Functioning 
2622 Change in Physiological Health 
2572 Change in Psychological Functioning 

2312 Change In Attitudes and Values 

2322 Change in Beliefs, Commitments, and Philosophy of Life 

2322 Change In Beliefs, Commitments, and Philosophy of Life 

2312 Change in Attitudes and Values 

2442 Change in Perception of Things 

2422 Change in Perception of Self 

2272 Change In Occupational and Employabllity Skills 

3102 Change In General Knowledge and Understanding 

3202 Change In Specialized KnOwledgeand Understanding 

2212 Chanaeln Academic Skills 

2272 Changei? Occupational and Employabllity Skills 

3102 Change In General Knowledge and Understanding 

3202 Chan?* ln Specialized KnOwledgeand Understanding 

2312 Change In Attitudes and Values 

2322 Change in Beliefs, Commitments, and Philosphy of Life 

2132 Change In Motivation or Drive Level 

2102 Change in Aspirations 

2332 Change In FeeMngs and Emotions 

2102 Change In Aspirations 

3312 Change in Research and Scholarship Knowledge «nd Understanding 
3312 Change In Research and Scholarship Knowledge and Understanding 
3312 Change in Research and Scholarship Knowledge and Understanding 

(Continued) 



'More than one Outcomes Structure category may pertain to a single NCHEMS Outcomes Inventory category In order to best conform to that category 
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OUTCOME CATEGORY OF 
THE NCHEMS INVENTORY (Continued) 


CORRESPONDING CATEGORY OR CATEGORIES OF THE 
OUTCOMES STRUCTURE TYPE-OF-OUTCOME DIMENSION (Continued) 


3 1.0 00 Community Development" 

3.1.0 01 Community Educational Development 
3.1.0 02 Faculty/Staff Educational Development 

320 00 Community Service" 

3 2 0 01 Extension Services 
3 2 0 02 Personal Services 

3 2 0 03 Extramural Cultural and Recreational Services 

3 2 0.04 Financial Impact on the Community 

3 3 0 00 LongerTerm Community Effects" 
3 3 0.01 Social Impact 
3.3.0.02 Economic Impact 


2002 Change in Human Characteristics 
2002 Change in Human Characteristics 
2002 Change In Human Characteristics 

4002 Resource and Service Provision Outcomes 
4202 Provision of Direct Services 
4202 Provision of Direct Services 
4102 Provision of Facilities and Events 
4202 Provision of Direct Services 
1000 Economic Outcomes 

0002 Change Outcomes 

2002 Human Characteristics Outcomes 

1000 Economic Outcomes 


" "The "audience" dimension would account for "community" and the "time" dimension of the Outcomes Structure would account for "longer term" 
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Appendix D 



Possible Copy for a Brochure or Flier Introducing 
Outcomes Concepts and the Outcomes Structure 



Introducing the NCHEMS Outcome Structure 

A Flexible, Practical Tool to Help Faculty and 
Administrators in Their Planning, Decision 
Making, and Communication about Educational 
Outcomes 

1. What Is an Educational Outcome in Post- 
secondary Education? 

Postsecondary education outcomes are the 
end results of the processes that occur within 
postsecondary education institutions and 
programs, such as instructional activities, 
research and scholarship activities, and com- 
munity service programs. They include both 
the direct results of these processes and any 
short-term and long-term consequences of 
those direct results. Some involve maintenance 
of skills, standard of living, and other individ- 
ual and social conditions, while others in- 
volve primarily change of different typeo. Fur- 
thermore, some of them may be intended out- 
comes while others are unintended. Although 
outcomes are generically neutral, people at- 
tach positive and/or negative values to them, 
and see them as performing or not performing 
particular useful functions for individuals, a 
particular community, particular aggregates of 
people (for example, "handicapped people"), 
or other entities such as the environment. One 
can focus on outcomes for students, for the 
local community, or for some other group. 

2. What Is the NCHEMS Outcomes Structure? 
The NCHEMS Outcomes Structure is a flexible, 
integrated, and comprehensive framework and 
system for organizing outcomes and outcomes 
information for purposes of classification, 
analysis, and decision making. It was 
developed by the NCHEMS staff over more 
than two years of concentrated effort, and 
resvs upon many significant bodies of previous 
work. Preliminary trial use and review of thi° 
S'aicture suggests that it is applicable to the 
full range of educational outcomes and has a 
number of quite practical uses for faculty and 
administrators on college and university cam- 
puses. 

3. Why Was the Outcomes Structure Developed? 



Without agreement on a common language 
and context for outcomes, it is difficult for in- 
stitutional faculty and staff to communicate 
succinctly— with each other across disci- 
plinary lines and with important publics— what 
their institution and/or program is accom- 
plishing and how it differs from its counter- 
parts. Such communication is especially 
important in this "age of accountability." Prior 
to now, many people in our institutions have 
found it uncomfortable to think and make 
plans in terms of outcomes, let alone talk con- 
cretely in such terms, preferring instead to 
focus exclusively on activities, process, and 
"process quality." Furthermore, the potential 
volume of outcomes information in post- 
secondary education is quite formidable, 
which poses a significant barrier to using out- 
come information in institutional and program 
planning and management. Although a number 
of attempts have Deen made in the past to 
develop such a system, none of them included 
the full range of possible outcomes (which 
limi'dd their applicability), they were often in- 
flexible, and in other ways they proved 
inadequate to meet some practical needs. 

4. What Practical Uses Does the Outcomes 
Structure Have? 

A. Communication— The terms and defini- 
tions in the Structure can serve as a work- 
able common language where the out- 
come concepts and relationships remain 
constant from one use of the outcomes in- 
formation to another, and it can thus 
facilitate effective communication about 
outcomes. 

B. Stimulation— The Structure can help create 
an awareness of postsecondary education 
outcomes and of the need for focusing on 
them, plus it can serve as a device that will 
stimulate people to focus on outcomes in a 
more systematic and concrete manner and 
to discuss them. 

C. Planning, Management, and Evaluation— 
The Structu'3 can serve as a tool to assist 
directly in he following functions: needs 
assessment, goal identification, goal 
assessment, goal translation, resource 
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5. 



allocation, program planning, evaluation, 
and outcomes information storage and 
retrieval. 

D. Research— The Structure can suggest gaps 
that need to be researched, research data 
bank applications, hypotheses for study, 
and so forth. 

The Outcomes Structure was designed to be of 
use, in the ways described above, at the in- 
stitutional and program level. However, a num- 
ber of faculty who reviewed the Structure and 
tried it out in a preliminary manner expressed 
the opinion that it also could be helpful to in- 
structors in planning individual courses. 
Several students and a student personnel 
worker felt it could be useful in helping stu- 
dents plan what they want to get out of college. 

What Is the Makeup of the Outcomes Struc- 
ture? 

The Outcomes Structure is a systematic and 
comprehensive way to categorize the out- 
comes of postsecondary education in terms of 
three "dimensions" and to relate outcomes to 
one another in a logical pattern, or to "struc- 
ture" them. 

The three dimensions are "audience," "type- 
of-outcome," and "time, M and they are 
illustrated graphically below: 



AUDIENCE 
DIMENSION 

(Who or what receives 
or is effected by the 
outcome or 15 iniend 
ed to receive or be 
affected by if) 



TIME DIMENSION 

(When is the out 
come expected to 
occur orwuendoes 
it occur'') 



TYPE OF OUTCOME DIMENSION 



(What basic entity is or <s intended to 
be maintained or changed'') 



The "audience" dimension is divided into the 
following five broad categories (which are in 
turn further divided into appropriate sub- 
categories): 

1. Individual/Group Clients 

2. Interest-Based Communities 

3. Geographic-Based Communities 

4. Aggregates of People 

5. Other Audiences 

The "type-of-outcome" dimension is also 



divided into five broad categories, each of 
which has additional subdivisions: 

1. Economic Outcomes 

2. Human Characteristic Outcomes 

3. Knowledge, Technology, and Art Form Out- 
comes. 

4. Resource and Service Provision Outcomes 

5. Other Maintenance and Change Outcomes 

The categories and subcategories needed for 
the "time" dimension vary according to the 
audience and type-of-outcome of concern. 
Therefore, formal categories and sub- 
categories are not included in the structure for 
this dimension. 

Standard definitions are provided for each 
category and subcategory of each dimension, 
and straightforward procedures are provided 
for "extending the Structure" in the cases 
where still more specificity and detail are 
needed. \ coding system is also provided, for 
thosu needing it. 

6. Where Can I Find More Information? 
The following two documents about the Struc- 
ture are available (from NOHEMS Publications, 
P.O. Drawer P, Boulder, Colorado 80302): 

A. A Structure for the Outcomes of Post- 
secondary Education — Presents six at- 
tributes of an educational outcome and five 
other factors that are important for under- 
standing particular outcomes, and provides 
detailed background discussion about 
each. Gives an in-depth description of the 
Structure, its rationale, the developmental 
principles and evaluative criteria used, its 
development, and its underlying empirical 
support. 

B. The Outcomes Structure: An Overview and 
Procedures for Applying It in Postsecondary 
Education Institutions— Provides an over- 
view description of the Structure, gives 
descriptions and examples of the various 
ways that the Structure can be used in a 
practical manner at postsecondary institu- 
tions; provides step-by-step procedures for 
three basic processes using the Structure, 
which can in turn be applied to a number of 
other tasks; and provides guidelines for 
orienting people on a campus to outcomes 
concepts and the Structure, and for training 
people to apply them. 
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For those who are interested, another 
document is also available that pro/ides a 
compilation of information about each of over 
80 previous efforts to "structure" educational 
outcomes and related concepts that were 
found in the literature: Previous Attempts to 
"Structure" Educational Outcomes and Out- 
come Related Concepts. If additional infor- 
mation or assistance is needed concerning the 
Structure, contact NCHEMS. 



Appendix E 

important Questions and Criticisms of the Outcomes Structure 
Posed by Particular Reviewers a 
and Responses by the Developers of the Structure 



1. By breaking outcomes into component types 
and ever more detailed categories, is it not 
possible to focus so much on specifics that 
one loses sight of the overall, combined whole 
which is more than the sum of the parts? This 
is a real danger, but it does not mean it is im- 
proper or lacks value to focus on the com- 
ponents that constitute the whole. Rather, it 
cautions us to constantly, as we examine a 
part, keep in mind the relationship of the part 
to the other parts and the whole. It is difficult, if 
not impossible, to adequately describe the 
whole without focusing also on each important 
component, for example, describing or 
analyzing what it means to be an "educated 
person." 

2. Could not this attempt to objectify outcomes 
and develop standard categories and 
definitions lead to a stifling of diversity, in- 
novation, and change— especially if the Struc- 
ture is used by administrators to spell out the 
outcomes desired and the process to meet 
those outcomes in such specific, precise terms 
that there is little room for innovativeness on 
the part of faculty members? Institutions and 
programs can be unique in the clientele served, 
the outcomes that are attempted or attained, 
and the means used to reach (hose outcomes. 
Preliminary use of the Structure shows that the 
audience dimension provides a comprehensive 
universe of categories of potential clients from 
which to choose, and that the type-of-outcome 
dimension provides a comprehensive universe 
of outcomes categories from which to choose. 
Thus, using the lists of categories as a 
checklist could stimulate consideration of 
specific unique audiences and types of out- 
comes that might not have been considered 
otherwise. 

Although administrators could use the Struc- 
ture to stifle innovation, they also could 
stimulate faculty to be more specific and con- 
crete in delineating the outcomes intended, 



and thus encourage consignation of alter- 
native ways of bringing about the outcomes 
based on the desired end results. The Out- 
comes Structure can clearly be used to help 
stimulate such concreteness in thinking, as 
demonstrated by the joint CASC/NCHEMS 
project. 

Would not the process of using the Structure 
take more of my time and energy than I can af- 
ford? To use the Structure as an aid to 
development of a complete program of goals, 
objectives, and priorities for a complex in- 
stitution and its programs is an extremely 
lengthy process. But that process would be 
lengthy whether or not the structure was used. 
It is possible that the systematic approaches 
developed for using the Structure, the 
definitions and concepts outlined, and use of 
the Structure as a checklist of the universe of 
alternative possible focuses can shorten the 
process appreciably. Furthermore, certain ap- 
plications of the Structure do not have to be 
time consuming to be rseful— for example, 
stimulating systematic and more concrete 
thinking by faculty and staff of what they are 
trying to accomplish in their programs and 
courses, of why they are trying to accomplish 
those ends in particular ways, and of how to 
show others that they are accomplishing their 
goals. Similarly, experiences at the University 
of Colorado suggest that one who is properly 
oriented to outcome concepts can use the 
Structure to evaluate the coverage of extensive 
lists of goals relatively quickly and e f ciently. 

Does not the use of psychological jargon for 
some terms in the Structure diminish its 
usefulness as an aid to communication? 
Jargon of any type should be avoided 
whenever possible, and an attempt was made 
to avoid its use in developing the Structure. But 
for certain outcomes, particularly student out- 
comes, psychological terminology appeared to 
be the most generic and descriptive, and the 



Reprinted from Chapter 4 of Oscar T Lenning, Yong S Lee, Sidneys Mlcek, and Allan L 
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most widely accepted. Council for the Advance* 
ment of Small College visitors to the cam- 
puses did perceive that the Structure could aid 
in "communicating outcomes across disci- 
plinary lines. * Nevertheless, it is to be hoped 
that extensive use of the Structure will sug- 
gest better terms in some areas of the 
Structure. 

5. Does not the fact that the code numbers are 
disconcerting to some people reduce the 
Structure's usefulness for them? Possibly, but 
it need not be so. Code numbers for each 
category were included for two purposes: (1) to 
give people a shorthand that would allow them 
to record outcomes and communicate them to 
knowledgeable others verbally or in writing in a 
more succinct and efficient manner, and (2) for 
use in outcomes information storage and 
retrieval systems. For other uses, the codes 
could be ignored. This is what happened in 
follow-up applications of the Structure at 
Spring Arbor College. Interestingly, in the 
CASC/NCHEMS interviews, when the respon- 
dents referred to particular outcome cate- 
gories, they generally referred to the code 
numbers rather than the category name— for 
example, "Outcome 2240" rather than "ex- 
pression and communication skill outcomes." 
Thus, some people who initially find the codes 
disconcerting may later become accustomed 
to using them and actually find them useful. 
But users of the Structure are encouraged to 
modify it as appropriate for best meeting their 
context and needs. 



6. The orientation of the Structure is foreign to 
the orientation of most educators— they just 
don't think this way. Would it not have been 
better and more acceptable to them if it were 
closer to their orientation in its form and con- 
tent? The authors tried to make the Struc- 
ture as generic, theoretically and philosoph- 
ically neutral, and comprehensive in its cover- 
age as possible, and this guided the develop- 
ment of the Structure. One person who raised 
this question suggested moving the know- 
ledge and understanding categories in front 
of the human characteristic outcome cate- 
gories, because "they are more important to 
most educators." But such a change would 
have made the Structure less neutral than 
retaining the categories in alphabetical order. 
The NCHEMS Inventory of Outcome Variables 
and Measures, which is the forerunner of the 
Outcomes Structure, was closer in its orien- 
tation to that of the typical educator who will 
use the Structure. However, that very charac- 
teristic led to criticism by some that many of 
the terms used were value-laden (it was not 
philosophically and theoretically neutral), that 
it was too selective in its coverage (not com- 
prehensive), and that it was too specific in its 
orientation and lacked some internal con- 
sistency (not generic). 



© - 

ERLC 



60 



65 



Appendix F 



Some Materials for Possible Use at 
the Campus Orientation Workshops 



Page 

1 . A Possible Workshop Agenda 62-63 

2. Sample Transparencies That Could Be Used in Presenting 

an Overview of Outcome Concepts and the Structure 64-73 

3. A Relevant Instrument for a Work Exercise 74-77 

4 A Relevant Role-Playing Scenario 78 
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Possible Workshop Agenda 4 



1. A short introduction that outlines why the 
campus may have a need to apply outcome 
concepts and the Outcomes Structure. 

2. An overview of outcomes concepts and the 
NCHEMS Outcomes Structure, using transpar- 
encies such as those included in this appen- 
dix. Using color, for example, different color 
shades of plastic for different transparencies, 
can add to their attractiveness and attention- 
getting quality. Be sure to encourage 
questions and requests for clarification during 
the presentation. 

3. A short overview summary of how the Structure 
could be used to develop lists of priority out- 
comes for one's program, using a handout in 
which are reprinted Figures 1-5 of this 
document, which show what came out of 
various stages in the outcome list develop- 
ment process for the political science program 
at Alpha College. 

4. Conduct a relevant work experience that will 
stimulate participants to focus on the intended 
or needed program coals they perceive, ways 
of reaching those goals, and how outcome 
success might be documented for their 
program of concern or responsibility. Have par- 
ticipants respond quickly to the Student Learn- 
ing Outcomes Questionnaire included in this 
appendix— or ? >dified version of it— in terms 
of the program tor which they are primarily 
concerned or responsible, and then have them 
discuss their responses with one another in 
previously assigned small groups. The en- 
closed questionnaire is a revised version of the 
form used in the CASC/NCHEMS campus in- 
terviews to stimulate discussion. As with the 
other workshop materials, it is imperative that 
the stimulation questionnaire decided on be 
tried out in advance and modified accordingly 
before being used at the workshop. For ex- 
ample, you may find that it should be abbre- 
viated (any of items 6-50 could be deleted easily) 
to fit into 'lie time slot allotted for it in the 
workshop. 



•The portion of each agenda item above that is typed m italics is what could be 
printed in the formal agenda sheet to be handed out tw workshop participants, 
along with the time frames for each 



5. Review of the work exercise, where it is to be 
shown how the use of the questionnaire and 
the follow-up conversation was in fact an ab- 
breviated application of the outcomes Struc- 
ture, discussed previously, to one part of 
program planning. 

6. Conduct a role-playing exercise, making use of 
the scenario included in this appendix, which 
has the intent of shifting the group's attention 
to institution-wide outcomes, and outcome- 
oriented planning at that level. People to play 
the roles should be selected very carefully and 
invitations to play the roles should be made 
(preferably in person, and they should receive 
copies of the scenario at this time) well in ad- 
vance of the workshop. Furthermore, an orien- 
tation session with the role players, that in- 
cludes distribution of appropriate materials 
about the Structure and its use, should be held 
ahead of time so that they can be prepared 
adequately to assume their roles. 

For the workshop session itself, it would be 
desirable to have them seated around a round 
table with large identification place cards that 
everyone else at the table can easily read. In 
the CASC/NCHEMS iryouts of this role-playing 
scenario, the actors had difficulty responding 
in terms of their own institution (which each 
v as asked to do) because the discussion kept 
referring to "Alpha College. " Therefore, the 
scenario sheet distributed probably should 
refer to your own institution by name (instead 
of Alpha College), and the discussion during 
the session then would also refer to your in- 
stitution by name. It will probably be desirable 
to keep the fictitious names (or others deemed 
more appropriate and entertaining for your 
group) for the role participants, however, to 
keep it obvious to the group at all times that 
this is a role-playing situation designed to be 
nonthreatening to everyone. (For this reason 
also, a requirement was made for the CASC/ 
NCHEMS workshops that none of the role 
players be playing their real-life role.) 

7. Discussion and critique of the role exercise, 
including: how it related to reality; problems 
and issues related to institution-wide out- 
comes; and desirable modes, procedures, and 
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times for institution-wide goal setting for this 
institution. One person should have been 
assigned ahead of time to take notes during 
the role playing, to make observations about it 
to get the discussion started, and to lead the 
discussion. 

8. Wrap-up session where the workshop leader 
summarizes what happened, ties together the 
different parts of the works h op into a 
meaningful whole, and attempts to end on a 
note that will stimulate further thinking and 
encourage support, action, and involvement 
for campus follow-up efforts in this area (this 
workshop is only meant to be a brief introduc- 
tion to the whole subject). 



The Six W's 

In order to really understand educational outcomes the following must be 
answered: 

• What are the characteristics and makeup of the educational outcome of 
concern? 

• Which institutional resources and activities are combined, and in which 
ways, to bring about the outcome? 

• For Whom is the outcome intended, or who actually received or was af- 
fected by it? 

• Why\N\\\, or did, the outcome occur? 

• Where will, or did, the outcome occur? 

• When will, or did, the outcome occur? 
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Eleven Factors That Are Useful in 
Answering the Six Questions 



What Outcomes? 

1. ti — Products, Events, and Conditions 

2. Gnange Status — Maintenance and Change 

3. Focus 

4. Neutrality 

5. Measurability 

6. Output/Impact 

Which Way to Obtain Them? 

7. Froducer/Facilitaicr 

Outcomes For Whom? 

8. Audience 

Why Will, or Did, They Occur? 

9. Intended/Unintended 

Where Will, or Did, They Occur? 

10. Functional A>ea 

When Will, or Did, They Occur? 

11. Time — Point in Time and Duration 
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WHICH OF THESE DRAWINGS 
SUGGEST PRIMARILY STUDENT 
IMPACTS AND WHICH SUGGEST 
PRIMARILY STUDEN T OUTPUTS 
FORACOLLEG"' 



GRADUATE 
SCHOOL 
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The NCHEMS Outcomes Structure is a 
system and framework designed to 
help people effectively organize and 
use information about the full range 

of postsecondary education outcomes, 
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Outcomes Structure 



AUDIENCE 
DIMENSION 

(Who receives or 
is affected by 
the outcome?) 




(When is the outcome 
expected to occur, or 
when does it occur?) 

TYPE-OF-OUTCOME 
DIMENSION 

(What basic entity is 
maintained or changed?) 
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The Major (First Level of Detail) 
Categories for the Audience and 
Type-of-Outcome Dimension* 



A. AUDIENCE DIMENSION 

1. Individual/Group Clients 

2. Interest-Based Communities 

3. Geographic-Based Communities 

4. Aggregates of People 

5. Other Audiences 



B. TYPE-OF-OUTCOME DIMENSION 

1. Economic Outcome- 

2. Human Characterise Outcomes 

3. Knowledge, Technology, and Art Form Outcomes 

4. Resource and Service Provision Outcomes 

5. Other Maintenance and Change Outcomes 
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and concrete level. 
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The Audience/Type- of -Outcome Matrix 
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Individual/Group 
Clients 
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Communities 
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Economic Human Knowledge, Resource and Other Maintenance 

Outcomes Characteristic Technology, Service and Change 

Outcomes and Art Form Provision Outcomes 
Outcomes Outcomes 
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AUDIENCE 



B.S. Graduates in 
Mechanical Engineering 



TYPE-OF-OUTCOME 




Percentage of Graduates 
Obtaining Their First 
Full-Time Job in the Field 
of Their Choice . . . 



TIME 



Within One Year 
after Graduation 
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Uses of the Outcomes Structure 



A. COMMUNICATION 

B. STIMULATION 

C. PLANNING, MANAGEMENT, AND EVALUATION 

1. Needs Assessment 

2. Goal Identification 

3. Goal Assessment 

4. Goal Translation 

5. Planning and Management 

6. Evaluation 




Context/ 
Condition 



Resources 



Activities 



Outcomes 



GO^ s 



0B JECT/ve S 



Intended 



Actual 



D. RESEARCH 
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In using the structure, it must be 
clearly differentiated whether the 
focus is on institution-wide outcomes 
or on program outcomes. It is unwise 
to try to focus on both types of out- 
comes at the same time when using the 
structure. 
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STUDENT LEARNING OUTCOMES QUESTIONNAIRE 

SMALL COLLEGE FORM 



Small colleges desire to have various impacts on their students. These consequences (outcomes) of 
college attendance that are planned to happen may or may not be expected to vary with the type of 
student. This questionnaire will gather information about the particular student outcomes that small- 
college administrators and faculty members are trying to bring about for: (1) the students throughout 
the college, or (2) the students in the college program area for which they have primary responsibil , ties 
Ar.y students completing the questionnaire should respond in terms of what they as students feel are the 
desirable outcomes for their college or their program at the college, and that should be aimed at by he 
college or program. For all respondents (administrators, faculty, or students), please decide whether 
you are going to respond in terms of college-wide outcomes or in terms of outcomes for a particular 
program area . Indicate your decision in Item 1 below, and then respond to the remainder of the question- 
naire accordingly, by checking (/) the appropriate boxp- and completing the blanks 



1. Please respond to this questionnaire in one of two ways; in terms of outcomes for the college as a 
whole, or in t s of outcomes for your program within the college. Choose the area most important 
to you; r 

1 2 

O College-wide outcomes £J Program-area outcomes 

2. If you answered "program-area outcomes" for Item 1, what program will you be referring to throuqhout 
the questionnaire? 



(1) 



(2-4) 



What ^s your classification? Check (/) one box and fill in the blank at the right. This information 
is needed so that we can describe the diversity of our survey samples. Such identification will be 
presented to others only in the form of group statistics. 

Administrator £j (What position? ) (6-8) 

i 

Department Head O (What department? ) 

Other Faculty £J (What department? ) 

i " ~ '~ ' ' 
Student O (What major? ) 



4. When considering student outcomes, either for your institution or for your program, would you rather 
distinguish between outcomes for student subpopulations , or can you focus meaningfully on desired 
outcomes for "all students 1- general"? That is, are outcomes significantly different for: resident 
vs. commuter; majors vs. non-majors; upper- vs. low -division, etc 7 „ 

O Yes O No 

5. If "yes," please indicate the two groups whose respective outcomes would be most important to plan 
for separately, in the spacas below: 

Group 1 -- Group 2 (9-10 

Items 6-55 list some possible student outcome categories. To the right of the page are two columns of 
marking boxes that will allow you to rate the importance of each outcome for the group or groups of students 

ITr^Yl *j! te 5L ln * I i t F n ?' L f you a J swer ? d "y es " for Item 4, only the Group 1 column should be used. Please 
choose from the following three categories in making your ratings, and code as indicated. 

ESSENTIAL (Student outcomes that must be a direct focus of planning for this institution or 
program) E 

IMPORTANT (Student outcomes that should be a direct focus of planning for this institution 
or program) j 
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NOT NEEDED (May be nice, but Is less than important as a focus of planning for this 
institution or program) \ N 
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Essential E 

Important I 

Not Needed N 



6. 

7. 
8. 
9. 
10. 

11. 
12. 
13. 
14. 
15. 

16. 
17. 
18. 
19. 
20. 

21. 
22. 
23. 
24. 
25. 

26. 
27. 
28. 
29. 
30. 

31 . 

32. 
33. 
34. 
35. 

36. 
37. 
38. 
39. 
40. 

41. 

42. 
43. 
44. 
45. 

46. 
47. 
48. 
49. 
50. 

51 . 
52. 
53 
54 
55 



Group 1 
E I N 

Academic skills L 7L Q 

Citizenship and family membership skil Is LJ LJ l=L\ 

Creativity skills L d, )=l 

Expression and communication skills L=l '< 'i 

Intellectual skills • V I N 

Interpersonal , leadership, and organizational skills £7 £7 £7 

Occupational and employabil ity skills LJ LJ LJ 

Physical and motor skills yJ L =L LJ 

General knowledge and understanding LJ I-J I-J 

Specialized knowledge and understanding LJ LJ LJ 

Desires, aims, and goals lJ EJ EJ 

Dislikes, likes, and interests LJ LJ ^ 

Motivation or drive level LJ LJ LJ 

Attitudes and values L=J y=J h=L 

Beliefs, commitments, and philosophy of life LJ LJ LJ 

Feelings and emotions, for example, appreciations and satisfaction. O £7 £7 

Mores, customs, and standards of conduct ■■■■■ LJ LJ LJ 

Perceptual awareness and perceptions of self, others, and things.. LJ LJ LJ 

Physical fitness, traits, and health l—J LJ LJ 

Adventurousness and initiative LJ l—J LJ 

Autonomy and independence £7 ^=Z, Ej=L. 

Dependability and responsibility : LJ LJ LJ 

Dogmatism-openmindedness; authoritariamsm-egalitarianism LJ LJ LJ 

Flexibility and adaptability 'JLJLJ 

Habits E I N 

Psychological functioning A7 /7 

Tolerance and persistence LJ LJ LJ 

Completion or achievement award LJ i-J LJ 

Credit recognition LJ I-J l—J 

Economic access, independence, and productivity LJ i—J l—J 

Image, reputation, or status £7 ^3. ^ 

Licensing and certification L=J LJ lj 

Obtaining a job or admission to a follow-up program LJ LJ LJ 

Power and/or authority ■ LJ IJ LJ 

Job, school, or life success (other than above) lj lj lj 

' ' E I N 

I 7 r~] r~i 

Adjustment to retirement L =L y=L y=L 

Affiliations LJ LJ lj 

Avocational and social activities and roles LJ LJ lj 

Career and vocational activities and roles LJ LJ LJ 

Citizenship activities and roles El N 

Family activities and roles ^ ^L. ^2. 

Friendships and relationships ■■ ■■ ■■■ • LJ LJ LJ 

New knowlpdge and understanding from research and scholarship LJ LJ LJ 

Research and scholarship products <-J LJ LJ 

Production or preservation of art forms and works /_/ I-J l-J 

n O O 

— o n o 

nan 

~ a n 1. 1 

— o a o 



Group 2 
E I 

o a 

o o 

n o 

n a 

n n 

E I 

n a 

n o 

a a 

a o 

n a 

E I 

n a 

n o 

n n 

a n 

a a 

E I 

n n 

a o 

n o 

n o 

n n 

E I 

o o 

n a 

n o 

n n 

n n 

E I 

o a 
n o 

£7 o 

a a 
n o 

E I 

£7 £7 

£7 £7 

£7 £7 

£7 £7 

£7 £7 

E I 

£7 £7 

£7 £7 

£7 £7 

£7 £7 

£7 £7 

E I 

£7 O 

£7 £7 

£7 £7 

£7 £7 

£7 £7 

E I 

£7 a 

fZJ 13 

Of £7 

/_/ LI 

£7 £7 



N 

£7 
£7 
£7 
£7 
£7 

N 

£7 
£7 
£7 
£7 
£7 
N 

£7 
£7 
£7 
£7 
£7 
N 

£7 
£7 
£7 
£7 
£7 
N 

£7 
£7 
£7 
£7 
£7 

N 

£7 
£7 
£7 
£7 
£7 

N 

£7 
£7 
£7 

fj 
£7 

N 

£7 
£7 
£7 

a 
a 

N 

£7 
£7 
£7 
£7 
£7 

N 

£7 
£7 
£7 
£7 
O 



NflTF- IF vntl ARF UNSURF WHAT ANY OF THE ABOVE OUTCOME CATEGORIES MEAN, DEFINITIONS AND EXAMPLES ARE PROVIDED 
K T ?HE MANUAL £5r THE STUD F NT LEARNING OUTCOMES QUESTIONNAIRE , AVAILABLE FROM THE CAMPUS COORDINATOR FOR THIS 
STUDY. 

7b 

SO 




For items 56-60, choose one of the outcome categories that you rated "essentia." an H l*,vi, „ *u 
most about prior to this time. Indicate this category in Se farge box Mot: * * th ° U9ht 



(62-6; 



56. 



57. 
58. 
59. 
60. 



(64-65 



5Scf.VlS2!u2 Jor e tSs C ou f ico m e nd c^S^ ° UtC ° meS f ° r the StUde " tS ° f that you would consider 

61. 

62. 

63. " 

64. " " " 

65. " ~ ~~~~ 



accountability" there are increasing 11^ » S ^ the outcome will happen. And in this "age of 
say they are bringing about have" ^faM occurred inThp^f 1 ^ that the ° UtCOmeS colle 9 es and P^ams 
observations or measures (including se r-reoorr measures hf- I'S ^ v I d ' d be J° w f ° r Items 56a-S5a indicate some 
major concrete ev^e nce that the Darticular i-rT?^ « limited to them) that you feel would constitute 

are, in-TaTt^cTuTrT^ or id occur for x mp S SudS '"ra'tin.;^ f1ed <n ItemS 56 ' 65 aS havinq 
actions or reactions to carefully planned stimS i' ncreased attend™ «lt° n a P art l cu1ar test or exam, observed 
written papers, and observing an oral demonstration by the studen!) • ***** ° f " mpUS eVentS ' 

56a. 

57a. "~" 

58a. 

59a. 

60a. 



ERIC 



76 



81 



62a. 
63a. 
64a. 
65a. 



In the blanks provided below for Items 56b-65b, identify some specific student activ ities and experiences 
you especially think would contribute to the realization of the particular student outcomes that you 
identified in Items 56-65 (for example, peer tutoring, participation in political campaigns, community 
service projects, organized in-class confrontation debates of selected issues, and small discussion groups) 



56b. 
57b. 
58b. 
59b. 
60b. 

61b. 
62b. 
63b. 
64b. 
65b. 



Age and sex groupings have often been found helpful in interpreting survey data. For Items 66 and 67, 
please check (/) the appropriate box. 

66. What is your sex? 

i * 

O Female [J Male ( 66 ) 

67. What is your age: 

[J Under 25 O 25-35 O 36-50 O Over 50 (67) 

For students, what is your year in school? 

O Freshman O Sophomore O Junior O Senior O Graduate or (68) 

Professional 
Student 



Modified Scenario for the 
Role-Playing Session during the 
CASC/NCHEMS On-Campus Workshop 



Roles 

Nick Knocks - College President (Committee Chairperson) 

Bill Grave — Vice-President for Finance 

Mavis Learned — Vice-President for Academic Afraiis 

Eldrige Bridger - Vice-President for Student Affairs 

Leroy Wowsm — Vice-President for Public Relations and Development 

Fredrick OTool — Faculty representative 

Mitzi Gainsmore — Student representative 

Frank Memory — Alumni representative 

Richard Green — Community leader and bank president 



The Setting 



The Jonathan Doe Fo nidation has made an offer 
to Alpha College to provide $25C,000 of un- 
solicited funds for s fr engthening the on going 
program of the college during the coming year. 
The only requirements are that it be W items in 
addition to those currently budgeted for next year, 
that it not involve facilities procurement or capital 
investment, and that the college notify the foun- 
datic • within six months exactly what the money 
will b~ used to do (if the college does not meet 
♦his requirement, the money will revert to one of 
the foundation's other charities). 

Upon receiving word of the good news, Alpha 
College's Board o? Trustees directed President 
Knocks to present them in three months with a 
priority list of student educational outcomes that 
the college should especially strive to improve, 
and recommendations for funded supplemental 
activities and for materials to bring about the 
desired improvements during the coming year. !n 
response to tnis request, President Knocks of 
Alpha med a nine-rrember commr ?e represent- 
ing th najor groups and areas of concern on 
campu-. vith himself as chairperson. In aoJition, 



an announcement was made to the campus 
community tnat anyone interested could sit in cn 
the meeting as an observer. (If an observer felt 
strongly enough about something being 
discuss3d, however, they would be allowed to 
pass their insight or concern on to the committee 
in a brief fashion.) 

At the committee's firs, meeting last week they 
discussed a new Outcomes Structure designed to 
help colleges decide on appropriate outcomes for 
their campus. They decided that special areas of 
concern were: aspirations; competence and skills; 
morale, satisfaction, and affective characteris- 
tics; perceptual characteristics; and psycho- 
logical adjustment characteristics. They also 
decided that at their next meeting they would 
focus on developing a priority list of concrete 
competence and skill outcomes, along with ways 
of improving those c .icrete outcomes. It was 
also decided that any discussion about the other 
four outcome areas woi:id have to wait until later 
meetings of the grcup. President Knocks is just 
now calling their second meeting to order. 
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